SlideShare a Scribd company logo
1 of 21
Download to read offline
White Paper




EMC BACKUP-AS-A-SERVICE: APPLIED BEST
PRACTICES FOR SERVICE PROVIDER BILLING
EMC Avamar and EMC Data Protection Advisor
   • Multi-tiered billing accurately reflects data processed
   • Automation streamlines reporting
   • Deduplication reduces redundant data




                   EMC Solutions Group

                   Abstract
                   This white paper discusses billing best practice recommendations for EMC®
                   Avamar® based Backup-as-a-Service (BaaS) solutions. The white paper details
                   the benefits and design considerations of different BaaS billing solutions, and
                   presents a new multi-tiered billing approach, and includes a review of how to
                   implement these billing solutions.

                   August 2012
Copyright © 2012 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its
publication date. The information is subject to change without notice.

The information in this publication is provided “as is.” EMC Corporation makes
no representations or warranties of any kind with respect to the information in
this publication, and specifically disclaims implied warranties of
merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this
publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation
Trademarks on EMC.com.

All trademarks used herein are the property of their respective owners.

Part Number: H10789




   EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing   2
                                     EMC Avamar and EMC Data Protection Advisor
Table of contents
  Executive summary ............................................................................................................................. 5
     Overview .......................................................................................................................................... 5
     Business case .................................................................................................................................. 5
     Solution overview ............................................................................................................................ 5
     Key results/ recommendations ........................................................................................................ 6

  Introduction ....................................................................................................................................... 7
     Purpose ........................................................................................................................................... 7
     Scope .............................................................................................................................................. 7
     Audience.......................................................................................................................................... 7
     Terminology ..................................................................................................................................... 7

  Backup-as-a-Service billing options.................................................................................................... 9
    Baseline billing recommendations ................................................................................................... 9
    Billing models .................................................................................................................................. 9
        Single-tiered primary protected data report ................................................................................. 9
        Back-end deduplicated capacity................................................................................................ 10
        Multi-tiered primary protected data report ................................................................................. 10
    Premium uplifts/billing options ..................................................................................................... 13

  Billing implementation ..................................................................................................................... 14
      Billing implementation ................................................................................................................... 14
          Avamar built-in reports .............................................................................................................. 14
          Avamar Postgres and MCS databases ........................................................................................ 14
          Data Protection Advisor ............................................................................................................. 14
          Avamar Management Console Command Line Interface............................................................. 14
          Primary protected data reports .................................................................................................. 15
          Back-end Capacity report .......................................................................................................... 16
          DPN Summary report ................................................................................................................. 16
          Multi-tiered primary protected data report ................................................................................. 18

  Automating reporting ....................................................................................................................... 19
     Automating reporting ..................................................................................................................... 19
     Using Avamar reports ..................................................................................................................... 19
     Using DPA ...................................................................................................................................... 19
     Creating reports through ODBC ...................................................................................................... 19

  Conclusion ....................................................................................................................................... 20
    Summary ....................................................................................................................................... 20
    Findings ......................................................................................................................................... 20


                                           EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing                                       3
                                                                                             EMC Avamar and EMC Data Protection Advisor
References ....................................................................................................................................... 21
   White papers ................................................................................................................................. 21
   Product documentation.................................................................................................................. 21




                                        EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing                                    4
                                                                                         EMC Avamar and EMC Data Protection Advisor
Executive summary
Overview            A service provider’s ability to successfully and competitively monetize an “as-a-
                    service” offering in the marketplace is one of the most critical aspects to its
                    operational success. The EMC® Avamar® backup solution is an enterprise backup
                    solution that is popular in the Backup-as-a-Service (BaaS) market. One of the reasons
                    for Avamar’s popularity is its deduplication efficiency.

                    Several options are:
                      •   Service provider charge each customer for deduplicated space utilization.
                      •   Customers are charged based on non-deduplicated space utilization.
                      •   Customer are charged based on the space being processed (also known as
                          protected) on their systems.

                    Service providers are using each of these options operationally. Which is the correct
                    approach for your business model? While this white paper cannot answer that
                    question, it outlines a variety of potential billing models that you can consider. The
                    discussion of each model includes its benefits and drawbacks along with a
                    discussion about how you can implement it.

Business case       Service providers that develop and deliver as-a-service solutions have a common
                    requirement for the billing processes. There must be an effective way to track, report,
                    and bill their customers based on the usage rates agreed to in the contracts. When it
                    comes to “standard” storage components and Storage-as-a-Service (STaaS) offerings,
                    this can be a straightforward process. Monthly bills are based on the amount of raw
                    storage consumed by the user and are determined from the base storage subsystem.

                    When you view BaaS, a key driver in cost reductions for the service provider is to use
                    solutions that employ data deduplication. This reduces the amount of data that
                    needs to be retained and stored by the system. This enables service providers to
                    dramatically reduce the amount of storage infrastructure needed to support the
                    backup processes yet still provide the same levels of recovery capabilities, which
                    yields significant cost savings. When you create a billing process, it is no longer a
                    simple one-to-one ratio of data to storage, because of the deduplication technology
                    used during the backup processes. Multiple backup agents may reference the same
                    data objects across datasets, which makes it difficult to count the effective used
                    storage.

                    Different types of data deduplicate at different rates, some higher and some lower.
                    When you couple these characteristics with the retention policies, you can add a lot
                    of variability into the storage usage profile.

Solution overview   Within the EMC data protection solutions, there are different billing methods that can
                    be supported based on the service provider’s needs, including:
                      •   Protected front-end primary data
                      •   Back-end deduplicated data
                      •   The types of data being stored and their deduplication efficiency


                             EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing     5
                                                                EMC Avamar and EMC Data Protection Advisor
•   Some combination of these methods

                  Different cost tiers can be created for different types of data based on:
                    •   How well data deduplicates
                    •   Retention periods
                    •   The amount of data a client sends to the service provider during a backup
                        session

                  The service provider must decide which model to use. It is important to understand
                  the benefits and risks for each billing model so you can plan the cost structures
                  appropriately to drive a profitable BaaS business.

Key results/      This solution provides several multi-tiered billing models for service providers that
recommendations   can:
                    •   Provide a more effective way to manage billing for customers
                    •   Reduce backup costs through different types of billing options
                    •   Simplify reporting by automating the process




                           EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing     6
                                                               EMC Avamar and EMC Data Protection Advisor
Introduction
Purpose        This white paper provides a detailed discussion of the different billing models and
               corresponding implementation options a service provider must consider when
               building a BaaS solution.

Scope          The scope of this white paper is to:
                 •   Show how a service provider can build a billing model for structured and
                     unstructured data
                 •   Provide overviews of typical billing models and introduce additional multi-
                     tiered models
                 •   Use Avamar’s automation mechanisms to generate billing data

Audience       This white paper targets the business and technical architects of service providers s,
               who are responsible for developing and implementing the billing aspects of a BaaS
               solution. It is expected that the reader will have baseline knowledge of Avamar, EMC
               Data Protection Advisor (DPA), and the terms associated with disk-based
               deduplication techniques.

Terminology    This white paper includes the following terminology.

               Table 1.     Terminology

                 Term                   Definition
                 Deduplication          Data deduplication is a method of reducing storage needs by
                                        eliminating redundant data. Only one unique instance of the data
                                        is retained on storage media. Redundant data is replaced with a
                                        pointer to the unique data copy.

                 Garbage collection     Improves write performance by eliminating the need to perform
                                        whole-block erasure before every write. Garbage collection works
                                        in the background and accumulates data blocks previously
                                        marked for deletion, performs a whole block erasure on each
                                        "garbage" block, and returns the reclaimed space for reuse by
                                        subsequent write operations.

                 Retention policies     The rules that are used to determine how long data is maintained
                                        on the system and available for recovery.

                 Service provider       A company in the IT industry that provides organizations with
                                        communications, storage, processing, or other services. Although
                                        the term service provider can refer to organizational subunits, it is
                                        generally used to refer to third-party or outsourced suppliers.

                 Structured data        Data that is identifiable because it is organized in a structure. The
                                        most common form of structured data or structured data records
                                        (SDR) is a database where specific information is stored, based on
                                        a method of columns and rows.

                 Tenant                 A tenant is one of potentially many unrelated customers that are
                                        being serviced by the service provider’s shared infrastructure.



                          EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing          7
                                                               EMC Avamar and EMC Data Protection Advisor
Term                Definition
Unstructured data   Data that has no identifiable structure. For example, images,
                    videos, email, documents, and text are all considered
                    unstructured data within a dataset.




       EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing    8
                                          EMC Avamar and EMC Data Protection Advisor
Backup-as-a-Service billing options
Baseline billing   It is important that you create the billing structure with a base cost derived for a
recommendations    particular base type of data with a specified base retention policy. This allows a low-
                   cost charge for customers who do not need advanced data capabilities, but provides
                   uplifts to cover the costs for customers who do require advanced data capabilities. As
                   outlined in this chapter, these premium uplifts can be structured separately.

                   Often, service providers want to use a simple single-price or primary-GB costing
                   model for their customers. What this means is that when a customer backs up data,
                   regardless of the data type in the dataset, the customer is charged at a common rate
                   based on a fixed retention policy. When creating the costing structure, the service
                   provider must make assumptions about the amount of data that will be unstructured
                   (such as data that deduplicates well) versus the amount of data that will be
                   structured (databases or highly unique file types such as JPGs that do not
                   deduplicate well).

                   If the data mix turns out to contain more structured data than what was assumed, the
                   monthly price point will not deliver positive profits. Alternatively, if the data mix is
                   more unstructured than planned, the data mix will return a better profit to the service
                   provider. Therefore, a single price tier for all types of primary source data is risky.

                   Before delving into the various use cases for Avamar billing implementations, the
                   following is a breakdown of what Avamar BaaS service providers are using:
                     •   Ninety-five percent use Avamar’s Bytes Protected report (also known as Primary
                         Protected report).
                     •   Four-point-nine percent use a combination of Avamar’s Bytes Protected report
                         with Avamar’s v_dpnsummary mod_sent value. This specifies how many bytes
                         were transmitted from a backup client to the Avamar grid for a specific backup
                         job.
                     •   Zero-point-one percent use Avamar’s Back-End Capacity report (this feature
                         was added in Avamar 5.1).

Billing models     This white paper outlines the following billing models:
                     •   Single-tier primary protected data report
                     •   Back-end deduplicated capacity
                     •   Multi-tiered primary protect data report (two- and three-tiered)

                   Single-tiered primary protected data report
                   This is the most popular billing mechanism implemented by service providers for
                   Avamar-based BaaS solutions. Avamar incorporates several mechanisms to report on
                   the amount of data being protected. In addition to being discussed in this white
                   paper, this report is detailed in the Reporting section of the EMC Avamar 6.1
                   Administration Guide. There are several variations of the Bytes Protected report,
                   which enable this report to run on a system-by-system basis with or without a date
                   range. Each of these reports will be discussed in further detail in subsequent sections
                   of this document.


                            EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing      9
                                                                 EMC Avamar and EMC Data Protection Advisor
You can report on primary protected data with Avamar alone or with Avamar and DPA.
Avamar standard reports provide basic primary-capacity reports and you can use
them as a starting point to develop custom SQL queries. This may be necessary
because the basic reports do not include the Avamar domain to which a particular
client belongs. Once the report is compiled, DPA can also provide this information,
without the use of custom SQL queries, and can be used to upload the data to a
billing system.

Back-end deduplicated capacity
The best way to create a billing process is to base it on the amount of data being
stored on the back end by each customer. Because we are using deduplicated
storage, there are going to be data objects shared across customers. This means that
you cannot simply account for each used byte of data. More accurately, you have to
count each used byte for each customer; otherwise, one customer is paying for
shared data objects from which other customers would benefit.

Within the Avamar solution, we used the Back-end Capacity report to determine the
amount of deduplicated storage used on the system by each customer. This takes
into account data deduplication within all the customers’ agents being protected by
the Avamar system. When you total the amount of data represented by the unique
data objects, you can provide the output of consumed deduplicated bytes.

Because different types of data deduplicate at different rates, you will not know how
much back-end storage is going to be used as a sole function of protected primary
data. This is not an issue with the Back-end Capacity report, as each customer is
charged based on back-end utilization. In addition, this report counts all the data
kept on the system for retention purposes. By summing up all deduplicated storage
along with the storage that is associated with retention, you can determine an
individual customer’s exact Avamar storage use.

While customers are aware that deduplication percentages vary greatly based on file
type, they cannot accurately detail what types of files they have. Because of this, it is
difficult for a service provider to gauge, during the sales process, how much storage
space a customer’s data will consume and be billed for until the end of the first
month’s billing cycle.

The Back-end Capacity report is not a simple real-time SQL query of the Avamar
database. The Back-end Capacity report involves a multi-step process that simulates
a selective data replication for each set of customer servers. Running this report can
take a significant amount of time and depends on the number of servers and
retention policies. Because this report details deduplicated storage for a specified set
of clients, it is best run on a per tenant basis.

Note    You cannot use DPA information in the Back-end Capacity report.

Multi-tiered primary protected data report
Because of the risks of having a single price tier for all data types, a better alternative
is to create a tiered pricing structure for different types of data. In general, a three-tier
model makes the most logical sense, but you can also use a two-tier model.




          EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing          10
                                               EMC Avamar and EMC Data Protection Advisor
The multi-tiered models have the following attributes:
  •   Two-tiered
         Structured data
         Unstructured data

  •   Three-tiered
         Structured data
         Unstructured data without exclusion
         Unstructured data with exclusion

The fundamental goal of these multi-tiered reports is to provide service providers with
the ability to separate unstructured and structured data.

For these reports, the unstructured and structured data will be determined by the type
of Avamar agent used to perform the backup. All OS agents back up unstructured
data, while agents for Microsoft SQL Server, Microsoft Exchange, Oracle, DB2,
Sybase, and so on, back up structured data.

A service provider must carefully consider the average change rates for structured and
unstructured data. The typical change rate for an OS agent client is three-tenths of
one percent per day, while the average change rate for a database is typically six
percent. This is a large difference, which has a significant impact on:
  •   Backup windows
  •   The amount of data required for retention windows
  •   The amount of network traffic a client consumes

When you compare the single-tier primary protected report against the two- and
three-tiered reports, the previous change-rate discussion becomes critical and
justifies why a multi-tiered reporting mechanism may be the best solution.

The difference between the two- and three-tiered reports provides a way to implement
exclusion lists into the backups. You can use the exclusion lists to separate data
types that:

  •   Incorporate compression, encryption, or other techniques
  •   Eliminate white space within the file
  •   Do not deduplicate or compress well

By providing a way to bill separately, based on these datafile types, you can better
manage the cost per GB.

Common file types for unstructured data includes 7Z, JPG, AVI, MOV, TGZ, and so on.
Unstructured data that deduplicates well and typically would not be in the exclusion
list, includes file types such as DOCX, PDF, DOC, PPT, PPTX, and so on.




          EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing     11
                                              EMC Avamar and EMC Data Protection Advisor
By using multi-tiered data reporting, you can specify different billing rates for the
different data types based on deduplication efficacy. By charging directly based on
usage for the different data types, the costs charged will cover the appropriate
costing structures. Reporting for this type of billing structure requires custom SQL
scripting through Avamar’s Open Database Connectivity (ODBC) interface. This
enables you to group the data based on the available fields maintained in the Avamar
database.
Combining this data provides the information required by the service provider’s
billing infrastructure to generate the appropriate invoices and display usage reports
for the different tenants.




         EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing    12
                                            EMC Avamar and EMC Data Protection Advisor
Premium           When delivering backup services, some customers require expanded capabilities to
uplifts/billing   support their business needs. The costs for premium services should be billed
options           separately or as a different type of Service Level Agreement (SLA) only to those
                  customers requiring the services. Table 2 describes the available premium services.
                  Table 2.        Premium services

                    Service                      Description
                    Data restores                Many service providers provide a specific number of data
                                                 restores during the monthly billing period. Restores above
                                                 that level incur a cost or restoration fee. This maintains
                                                 control over customers who misuse the backup system to
                                                 provide disaster recovery services by performing daily server
                                                 restores to an alternate location.

                    Data output to tape          Using extended retention, through a media access node, data
                                                 can be written to tape on a defined, periodic basis (generally
                                                 only on a monthly basis), which provides longer-term
                                                 retentions on lower-cost media. The cost to write to tape is
                                                 based on price/GB written, and can be tiered based on the
                                                 data amounts that are written.

                    Longer retention policies    The service provider can increase the price/GB or percentage
                                                 uplift of the base price/GB charge to cover a longer retention
                                                 policy on a disk. Typical Avamar BaaS deployments use a 30-
                                                 day retention period. You must consider the implications of
                                                 longer retention periods in storage space utilization. EMC
                                                 recommends you assume a three-tenths of one percent daily
                                                 change rate for file server type workloads and a six percent
                                                 daily change rate for application-specific backups.

                    Replication                  If customers want their data replicated offsite, there is an
                                                 additional price/GB to cover this service. The service provider
                                                 can offer this type of service in a variety of ways. Replicating a
                                                 customer’s Avamar system to a service provider’s Avamar
                                                 system in a different geographic area has recently been
                                                 generating interest. Another alternative is to provide
                                                 customers with the ability to replicate their data from one of
                                                 the service provider’s data centers to another data center
                                                 located in a different geographic area.




                              EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing            13
                                                                   EMC Avamar and EMC Data Protection Advisor
Billing implementation
Billing          Depending on your environment and operational requirements, you have several
implementation   options for generating the Avamar billing reports discussed in the previous chapter.

                 You can use Avamar to generate billing-related information based on your
                 requirements.
                 Avamar built-in reports
                 Avamar provides many built-in reports that you can generate and modify directly from
                 the Avamar Management Console. You can customize these reports to create and
                 gain access to customer-specific data. It is also possible to use DPA to provide
                 additional reporting capabilities for Avamar. Despite the capabilities of Avamar and
                 DPA, neither provides the ability to generate new reports that combine information
                 from existing Avamar reports.

                 Avamar Postgres and MCS databases
                 Avamar uses a Postgres database to track of all its activities. The Management
                 Console Server (MCS) datafile name is mcdb and is located on the utility node in the
                 /usr/local/Avamar/var/mc/server_data/postgres folder. The MCS database provides
                 read-only access through defined public views and tables. You can access the
                 information in this database using custom SQL calls through ODBC. You can create
                 highly customized reports when you access mcdb information.

                 Avamar provides several MCS database views. These public views are documented in
                 Appendix C of the EMC Avamar 6.1 Administration Guide. You can access some views
                 through the Avamar management GUI. For example, the Backup Statistics screen
                 provides statistics by using data from the v_activities_2 database view.

                 The v_reports view lists all the MCS database views, along with the SQL calls used to
                 generate them. You can find additional information about this view in Appendix C of
                 the EMC Avamar 6.1 Administration Guide.

                 We used this view into the mcdb to generate the customer queries needed to create
                 several reports discussed in this white paper.

                 Data Protection Advisor
                 DPA provides a mechanism to automate the collection, analysis, and reporting of
                 Avamar grid usage by tenant, and can be used to customize many of the reports that
                 Avamar provides, without creating custom SQL queries.

                 Avamar Management Console Command Line Interface
                 The Avamar Management Console Command Line Interface (MCCLI) provides an
                 essential capability to the successful integration of Avamar into a service provider’s
                 portal. During the development of our reference portal and BaaS implementation, we
                 used the MCCLI extensively.

                 Note   MCCLI is not built as a reporting tool and has no mechanism to create, edit, or
                        run Avamar reports.




                          EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing    14
                                                             EMC Avamar and EMC Data Protection Advisor
Previous sections of this document have discussed the various types of reports that
are either used by service providers or that EMC recommends while designing a BaaS
solution. Each of those reports, in addition to one hybrid, is described in this section.

Primary protected data reports
Billing the customer based on the amount of data protected by the BaaS solution is
the most popular billing mechanism in production. For more information about this
billing method see Chapter 8 of the EMC Avamar 6.1 Administration Guide. There are
several variations of the Bytes Protected reports, which enable these reports to run on
a system-by-system basis with or without a date range.

These reports provide the maximum number of bytes protected for each customer.
The Activities – Bytes Protected Client report provides the maximum number of bytes
protected by the Avamar system in the last two weeks for each client.

Alternatively, the Activities – Bytes Protected Client – 2 report specifies the same
information for activity that occurred during a specified date range. You can run either
report on a daily basis or once per month. Service providers who run the report on a
daily basis typically do so to:
  •   Provide daily updates to the self-service portal
  •   Use the data to obtain the maximum number of bytes protected during a 30-day
      period, for which they bill the customer

Recommendations
EMC recommends that you run the Activities – Bytes protected Client – 2 report for
the preceding month on a daily basis, and update the self-service portal based on the
results of that report.

EMC also recommends, though not strictly a billing issue, that you display the output
of the Activities – No Activities and Activities – No Checkins information on the self-
service portal. This ensures that the customer is aware of systems that have not been
backed up recently.

These two reports do not include any storage used on the Avamar grid for retention
purposes. EMC recommends that you build a rate that includes the length of retention
for the Avamar infrastructure into the price per GB protected.

If you are going to bill a tenant based on the quantity of primary protected storage,
EMC recommends that you bill the customer using one of the following models:
  •   Run the Activities – Bytes Protected 2 report for the preceding month to obtain
      the largest amount of data protected on each system.
  •   Bill based on an average of the Activities – Bytes Protected report run on a daily
      basis for each system. If run at the root domain, the output of the Bytes
      protected Client and the Bytes Protected Client 2 reports list each server that
      was backed up and the corresponding amount of storage protected.




         EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing       15
                                             EMC Avamar and EMC Data Protection Advisor
Creating a per-tenant (Avamar domain) report requires additional work because the
report lists the servers, not the Avamar domains, to which they belong. To collect this
data on a per-tenant basis:
  •     Use a report template (for example, Clients Report template) that includes the
        client domain in the report.
  •     Use the Activities Report template that states that the client’s domain can be
        included.
  •     Customize the report to include each tenant instead of the root domain.
  •     From the report, extract the client name and, using Postgres/ODBC calls, map
        the name to its domain. You can use a view such as v_group_members, which
        includes client_name and full_client_name.

Back-end Capacity report
The Back-end Capacity report provides information on the deduplicated storage
usage by systems or tenants. The Back-end Capacity report performs a simulated
replication operation of an empty Avamar grid for the specified systems or tenants.
From this report, you can determine the exact Avamar grid usage and include any
deduplication benefits of the specified targets. This information includes all retention
data and any usage of globally deduplicated items.

Some service providers script the Back-end Capacity report to run once a month for
all their tenants. By scripting the Back-end Capacity report to run each tenant
sequentially, the variability in how long each run takes is irrelevant. While we feel this
is appropriate for small- and medium-sized Avamar BaaS deployments, this solution
is not yet highly scalable and should not be deployed in large BaaS environments
with hundreds of tenants.

Notes

         • With Avamar 6.0 and 6.1, you cannot use this report to determine the
           back-end capacity used by a system being backed up to an EMC Data
           Domain® system.
         • Only one instance of this report can be run at a time, requiring individual
           runs for each domain or client grouping. Only one instance of the Back-end
           Capacity report can be run simultaneously, because the system has to
           parse through all its metadata to determine how much of the back-end
           capacity is being used by the targeted systems.

DPN Summary report
You can use the DPN Summary report for billing through Avamar’s MCS report,
Activities – DPN Summary, or by directly accessing the v_dpnsummary MCS database
view through ODBC calls. You can use the information available from the DPN
Summary report by itself or combined with other report data, such as the Primary
Protected report. Table 3 describes the data available in the DPN Summary report.




           EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing      16
                                              EMC Avamar and EMC Data Protection Advisor
Table 3.       DPN summary report

  Variable name          Description
  Host                   The client hostname

  Mod_sent               The number of bytes of files that were modified since the last backup
                         and sent to the Avamar grid for protection

  Modnotsent             The number of bytes of files that were modified since the last backup
                         but were not sent to the Avamar grid due to global deduplication

  Totalbytes             Number of bytes processed

Note       You can use the data in this report to generate the Primary Protected dataset
           size on a per-host basis.

By running the DPN Summary report for a specified monthly billing period and
summarizing mod_sent, you can calculate the amount of data a client has sent to the
Avamar grid. However, this number does not include deduplication. To collect
deduplication information, the modnotsent variable needs to be used so that it is
possible to distinguish between what Avamar is protecting and what is captured in
the primary protected reports.

The report does not consider data that has been expired on the Avamar grid because
of the report’s retention policies. In addition, the report does not consider the amount
of Avamar space being used for data retention.

Example
As an example of the challenges inherent with using the mod_sent method for billing,
consider the use case of a BaaS environment in which a customer backs up a 1 TB
database. Assuming reasonable deduplication efficiency, the Avamar client sends
300 GB to the Avamar grid for the initial seeding of this file on day one. You can also
assume that data retention is set to 30 days and that there is a three percent day-to-
day change rate on the file. Once the initial seeding of 300 GB is complete, only 3 GB
is sent on a daily basis. You have a total mod_sent of 387 GB = 300 GB + (29 * 3 GB)
for the first month. That also equals the usage of the grid from a retention period,
which is reasonable.

The concern for the service provider occurs in month two, where only the change-rate
data is sent. The mod_sent for month two is 90 GB = (30 * 3 GB). The customer is
satisfied, while the service provider is not, as the total usage of the backed Avamar
grid, at the end of month two is 387 GB = 300 GB + (29 * 3 GB).

Note       There is more post processing required of DPN summary reports as they report
           on a per-host basis. The tenant needs to summarize this before uploading to
           the billing system.

Custom reports
You may want to generate custom reports based on a combination of primary
protected and mod_sent data. One typical use case that tracks mod_sent is to use
that information to validate what the daily change rate is for a particular customer.
You can do this by tracking mod_sent in combination with totalbytes and
modnotsent. Taken together, you can use totalbytes and modnotsent to determine


            EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing         17
                                                 EMC Avamar and EMC Data Protection Advisor
the daily change rate for systems and tenants. By having this change rate data, you
can fine tune the billing assumptions around the quantity of storage used for data
retention.

Multi-tiered primary protected data report
EMC recommends a custom multi-tiered billing model as an alternative to a higher-
risk single-price approach to billing. The recommended tiers are:
  •   Unstructured data with exclusions of specific, low deduplicatable data types,
      whose typical deduplication efficiencies are relatively low
  •   Unstructured data with no exclusions types, whose typical deduplication
      efficiencies are typically very high
  •   Structured datasets

There are no existing reports that collect this information, so you must create a
custom report using ODBC access to the Avamar database (MCDB), then using the
database views. You must collate and collect information from individual backup
jobs, including:
  •   The type of agent plug-ins used for the backups (OS agents versus application
      agents)
  •   The size of the file
  •   Dataset names (identify which datasets have exclusions versus no exclusions)
  •   Customer domain as set up within Avamar

By collecting this information for each back-up job, you can determine:
  •   The amount of data, by data type, that was sent to the Avamar grid
  •   The type of agent plug-in used
  •   The customer domain

You can then group the data, according to typical deduplication efficiency and plug-in
used, to be billed back to the customer. Custom SQL scripting through Avamar’s
ODBC interface is required to collate and organize the information. Once collected,
the information is forwarded to your billing infrastructure to generate the appropriate
invoices and display usage reports for the different tenants.




         EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing     18
                                             EMC Avamar and EMC Data Protection Advisor
Automating reporting
Automating         In order for you to cost-effectively scale out an as-a-service offering, automation is a
reporting          key requirement. The recommendations are not specific to any of the reports
                   previously listed; they are a general overview of what automation options are
                   available.
Using Avamar       You can create Avamar reports using the Avamar Admin > Tools > Manage Reports
reports            dialog box, and specify the attributes desired. You can create this report at the root
                   domain to include all Avamar domains or a separate instance of this report for each
                   Avamar domain. Your environment and the amount of post processing you wish to
                   perform will dictate which method to use.

                   Creating a mechanism within Avamar to automate the execution of a report requires
                   that you use a custom event profile. For more information, refer to the sections
                   “Creating a customer event profile” and “Events, Notifications, and Profiles” in the
                   Avamar Admin Guide.
                   As part of the event profile creation process, you select a triggering event code. EMC
                   recommends that you use an event code such as “garbage collection initiated” or
                   “garbage collection completed” as the trigger.

                   The automated mechanism requires that the reports be emailed. EMC recommends
                   that you email the reports to a service-provider-controlled mailbox that saves the
                   attachments to a local file, then you can process the reports as required.

Using DPA          You can create the same reports using DPA. DPA provides additional reporting
                   capabilities without customer scripts. It also provides more capabilities for
                   automating report generation than Avamar.

Creating reports   The most powerful and flexible automation option is to access the data from
through ODBC       Avamar’s database directly. You create the report through ODBC/SQL calls against
                   the Avamar Postgres database. Appendix C of the EMC Avamar 6.1 Administration
                   Guide provides a list of precreated views into the Avamar database. Avamar uses
                   many of these views for the prepackaged Avamar reports.
                   For example, the v_dpnsummary MCS database view includes the underlying data
                   used to create the Activities – Bytes Protected Client reports. The critical data is host,
                   activity, and totalbytes. Totalbytes provides the number of bytes processed on the
                   client.




                             EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing       19
                                                                 EMC Avamar and EMC Data Protection Advisor
Conclusion
Summary      There must be an effective way to track, report, and bill service providers’ customers
             based on the usage rates agreed to in their contracts. Different types of data
             deduplicate at different rates, some higher and some lower. When you couple these
             characteristics with the retention policies, you can add a lot of variability into the
             storage usage profile. The financial model and reports used by the service provider
             must accurately capture and bill customers based on their usage.

             This white paper provided an overview of several multi-tiered billing models for use
             with Avamar-based BaaS solutions. These multi-tiered reports provide service
             providers with a flexible set of billing alternatives that can be customizes for their
             specific environment.

Findings     This solution provides several multi-tiered billing models that can:
               •   Streamline data backup and restoration by reducing wasted storage
               •   Provide a more effective way to manage billing for customers
               •   Simplify reporting by automating the process




                      EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing     20
                                                          EMC Avamar and EMC Data Protection Advisor
References
White papers    For more information, see the following white papers:
                  •   EMC Backup-as-a-Service: EMC Avamar, EMC Data Protection Advisor and EMC
                      Homebase
                  •   EMC Avamar Technical Deployment Considerations for Service Providers
                  •   EMC Avamar Business Deployment Considerations for Service Providers
                  •   Creating “Backup as a Service” (BaaS) Solutions Leveraging EMC Avamar

Product         For more information, see the following product documents:
documentation
                  •   EMC Avamar 6.1 Administration Guide




                         EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing   21
                                                           EMC Avamar and EMC Data Protection Advisor

More Related Content

Viewers also liked

database recovery techniques
database recovery techniques database recovery techniques
database recovery techniques Kalhan Liyanage
 
Presentation on backup and recoveryyyyyyyyyyyyy
Presentation on backup and recoveryyyyyyyyyyyyyPresentation on backup and recoveryyyyyyyyyyyyy
Presentation on backup and recoveryyyyyyyyyyyyyTehmina Gulfam
 
Ict policy for Networked Society - GCC Dec 2013
Ict policy for Networked Society - GCC Dec 2013Ict policy for Networked Society - GCC Dec 2013
Ict policy for Networked Society - GCC Dec 2013Rene Summer
 
1948 creation of israel wed
1948 creation of israel wed1948 creation of israel wed
1948 creation of israel wedTravis Klein
 
Material movingirls
Material movingirlsMaterial movingirls
Material movingirlsMartaMuros
 
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...EMC
 
福音的開始、中斷與振興 景高
福音的開始、中斷與振興 景高福音的開始、中斷與振興 景高
福音的開始、中斷與振興 景高zenixls2
 
Studiu de piata imobiliara apartamente noi bucuresti
Studiu de piata imobiliara apartamente noi bucurestiStudiu de piata imobiliara apartamente noi bucuresti
Studiu de piata imobiliara apartamente noi bucurestiCostin Ciora
 
Agatha's Cover Letter for Continuum
Agatha's Cover Letter for ContinuumAgatha's Cover Letter for Continuum
Agatha's Cover Letter for ContinuumAgatha Yu
 
цахим хичээлийн зөвлөмж
цахим хичээлийн зөвлөмжцахим хичээлийн зөвлөмж
цахим хичээлийн зөвлөмжpvsa_8990
 
Forrester: How Organizations Are Improving Business Resiliency with Continuou...
Forrester: How Organizations Are Improving Business Resiliency with Continuou...Forrester: How Organizations Are Improving Business Resiliency with Continuou...
Forrester: How Organizations Are Improving Business Resiliency with Continuou...EMC
 
Copyright enforcement for the digital age q1 2013
Copyright enforcement for the digital age q1 2013Copyright enforcement for the digital age q1 2013
Copyright enforcement for the digital age q1 2013Rene Summer
 

Viewers also liked (19)

database recovery techniques
database recovery techniques database recovery techniques
database recovery techniques
 
Presentation on backup and recoveryyyyyyyyyyyyy
Presentation on backup and recoveryyyyyyyyyyyyyPresentation on backup and recoveryyyyyyyyyyyyy
Presentation on backup and recoveryyyyyyyyyyyyy
 
Fundamental, Applied and Action Research
Fundamental, Applied and Action ResearchFundamental, Applied and Action Research
Fundamental, Applied and Action Research
 
Ict policy for Networked Society - GCC Dec 2013
Ict policy for Networked Society - GCC Dec 2013Ict policy for Networked Society - GCC Dec 2013
Ict policy for Networked Society - GCC Dec 2013
 
1948 creation of israel wed
1948 creation of israel wed1948 creation of israel wed
1948 creation of israel wed
 
Material movingirls
Material movingirlsMaterial movingirls
Material movingirls
 
Antarctica music
Antarctica musicAntarctica music
Antarctica music
 
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...
Improve Patient Care and Reduce IT Costs with Vendor Neutral Archiving and Cl...
 
Informe de empleados
Informe de empleadosInforme de empleados
Informe de empleados
 
福音的開始、中斷與振興 景高
福音的開始、中斷與振興 景高福音的開始、中斷與振興 景高
福音的開始、中斷與振興 景高
 
Renaissance art
Renaissance artRenaissance art
Renaissance art
 
Formulario clientes
Formulario clientesFormulario clientes
Formulario clientes
 
Studiu de piata imobiliara apartamente noi bucuresti
Studiu de piata imobiliara apartamente noi bucurestiStudiu de piata imobiliara apartamente noi bucuresti
Studiu de piata imobiliara apartamente noi bucuresti
 
Agatha's Cover Letter for Continuum
Agatha's Cover Letter for ContinuumAgatha's Cover Letter for Continuum
Agatha's Cover Letter for Continuum
 
цахим хичээлийн зөвлөмж
цахим хичээлийн зөвлөмжцахим хичээлийн зөвлөмж
цахим хичээлийн зөвлөмж
 
Forrester: How Organizations Are Improving Business Resiliency with Continuou...
Forrester: How Organizations Are Improving Business Resiliency with Continuou...Forrester: How Organizations Are Improving Business Resiliency with Continuou...
Forrester: How Organizations Are Improving Business Resiliency with Continuou...
 
A tiny voice inside
A tiny voice insideA tiny voice inside
A tiny voice inside
 
Mex rev
Mex revMex rev
Mex rev
 
Copyright enforcement for the digital age q1 2013
Copyright enforcement for the digital age q1 2013Copyright enforcement for the digital age q1 2013
Copyright enforcement for the digital age q1 2013
 

More from EMC

Cloud Foundry Summit Berlin Keynote
Cloud Foundry Summit Berlin Keynote Cloud Foundry Summit Berlin Keynote
Cloud Foundry Summit Berlin Keynote EMC
 
EMC GLOBAL DATA PROTECTION INDEX
EMC GLOBAL DATA PROTECTION INDEX EMC GLOBAL DATA PROTECTION INDEX
EMC GLOBAL DATA PROTECTION INDEX EMC
 
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIO
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIOTransforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIO
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIOEMC
 
Citrix ready-webinar-xtremio
Citrix ready-webinar-xtremioCitrix ready-webinar-xtremio
Citrix ready-webinar-xtremioEMC
 
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES EMC
 
EMC with Mirantis Openstack
EMC with Mirantis OpenstackEMC with Mirantis Openstack
EMC with Mirantis OpenstackEMC
 
Modern infrastructure for business data lake
Modern infrastructure for business data lakeModern infrastructure for business data lake
Modern infrastructure for business data lakeEMC
 
Force Cyber Criminals to Shop Elsewhere
Force Cyber Criminals to Shop ElsewhereForce Cyber Criminals to Shop Elsewhere
Force Cyber Criminals to Shop ElsewhereEMC
 
Pivotal : Moments in Container History
Pivotal : Moments in Container History Pivotal : Moments in Container History
Pivotal : Moments in Container History EMC
 
Data Lake Protection - A Technical Review
Data Lake Protection - A Technical ReviewData Lake Protection - A Technical Review
Data Lake Protection - A Technical ReviewEMC
 
Mobile E-commerce: Friend or Foe
Mobile E-commerce: Friend or FoeMobile E-commerce: Friend or Foe
Mobile E-commerce: Friend or FoeEMC
 
Virtualization Myths Infographic
Virtualization Myths Infographic Virtualization Myths Infographic
Virtualization Myths Infographic EMC
 
Intelligence-Driven GRC for Security
Intelligence-Driven GRC for SecurityIntelligence-Driven GRC for Security
Intelligence-Driven GRC for SecurityEMC
 
The Trust Paradox: Access Management and Trust in an Insecure Age
The Trust Paradox: Access Management and Trust in an Insecure AgeThe Trust Paradox: Access Management and Trust in an Insecure Age
The Trust Paradox: Access Management and Trust in an Insecure AgeEMC
 
EMC Technology Day - SRM University 2015
EMC Technology Day - SRM University 2015EMC Technology Day - SRM University 2015
EMC Technology Day - SRM University 2015EMC
 
EMC Academic Summit 2015
EMC Academic Summit 2015EMC Academic Summit 2015
EMC Academic Summit 2015EMC
 
Data Science and Big Data Analytics Book from EMC Education Services
Data Science and Big Data Analytics Book from EMC Education ServicesData Science and Big Data Analytics Book from EMC Education Services
Data Science and Big Data Analytics Book from EMC Education ServicesEMC
 
Using EMC Symmetrix Storage in VMware vSphere Environments
Using EMC Symmetrix Storage in VMware vSphere EnvironmentsUsing EMC Symmetrix Storage in VMware vSphere Environments
Using EMC Symmetrix Storage in VMware vSphere EnvironmentsEMC
 
Using EMC VNX storage with VMware vSphereTechBook
Using EMC VNX storage with VMware vSphereTechBookUsing EMC VNX storage with VMware vSphereTechBook
Using EMC VNX storage with VMware vSphereTechBookEMC
 
2014 Cybercrime Roundup: The Year of the POS Breach
2014 Cybercrime Roundup: The Year of the POS Breach2014 Cybercrime Roundup: The Year of the POS Breach
2014 Cybercrime Roundup: The Year of the POS BreachEMC
 

More from EMC (20)

Cloud Foundry Summit Berlin Keynote
Cloud Foundry Summit Berlin Keynote Cloud Foundry Summit Berlin Keynote
Cloud Foundry Summit Berlin Keynote
 
EMC GLOBAL DATA PROTECTION INDEX
EMC GLOBAL DATA PROTECTION INDEX EMC GLOBAL DATA PROTECTION INDEX
EMC GLOBAL DATA PROTECTION INDEX
 
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIO
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIOTransforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIO
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIO
 
Citrix ready-webinar-xtremio
Citrix ready-webinar-xtremioCitrix ready-webinar-xtremio
Citrix ready-webinar-xtremio
 
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES
 
EMC with Mirantis Openstack
EMC with Mirantis OpenstackEMC with Mirantis Openstack
EMC with Mirantis Openstack
 
Modern infrastructure for business data lake
Modern infrastructure for business data lakeModern infrastructure for business data lake
Modern infrastructure for business data lake
 
Force Cyber Criminals to Shop Elsewhere
Force Cyber Criminals to Shop ElsewhereForce Cyber Criminals to Shop Elsewhere
Force Cyber Criminals to Shop Elsewhere
 
Pivotal : Moments in Container History
Pivotal : Moments in Container History Pivotal : Moments in Container History
Pivotal : Moments in Container History
 
Data Lake Protection - A Technical Review
Data Lake Protection - A Technical ReviewData Lake Protection - A Technical Review
Data Lake Protection - A Technical Review
 
Mobile E-commerce: Friend or Foe
Mobile E-commerce: Friend or FoeMobile E-commerce: Friend or Foe
Mobile E-commerce: Friend or Foe
 
Virtualization Myths Infographic
Virtualization Myths Infographic Virtualization Myths Infographic
Virtualization Myths Infographic
 
Intelligence-Driven GRC for Security
Intelligence-Driven GRC for SecurityIntelligence-Driven GRC for Security
Intelligence-Driven GRC for Security
 
The Trust Paradox: Access Management and Trust in an Insecure Age
The Trust Paradox: Access Management and Trust in an Insecure AgeThe Trust Paradox: Access Management and Trust in an Insecure Age
The Trust Paradox: Access Management and Trust in an Insecure Age
 
EMC Technology Day - SRM University 2015
EMC Technology Day - SRM University 2015EMC Technology Day - SRM University 2015
EMC Technology Day - SRM University 2015
 
EMC Academic Summit 2015
EMC Academic Summit 2015EMC Academic Summit 2015
EMC Academic Summit 2015
 
Data Science and Big Data Analytics Book from EMC Education Services
Data Science and Big Data Analytics Book from EMC Education ServicesData Science and Big Data Analytics Book from EMC Education Services
Data Science and Big Data Analytics Book from EMC Education Services
 
Using EMC Symmetrix Storage in VMware vSphere Environments
Using EMC Symmetrix Storage in VMware vSphere EnvironmentsUsing EMC Symmetrix Storage in VMware vSphere Environments
Using EMC Symmetrix Storage in VMware vSphere Environments
 
Using EMC VNX storage with VMware vSphereTechBook
Using EMC VNX storage with VMware vSphereTechBookUsing EMC VNX storage with VMware vSphereTechBook
Using EMC VNX storage with VMware vSphereTechBook
 
2014 Cybercrime Roundup: The Year of the POS Breach
2014 Cybercrime Roundup: The Year of the POS Breach2014 Cybercrime Roundup: The Year of the POS Breach
2014 Cybercrime Roundup: The Year of the POS Breach
 

Recently uploaded

Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 

White Paper: EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing - EMC Avamar and EMC Data Protection Advisor

  • 1. White Paper EMC BACKUP-AS-A-SERVICE: APPLIED BEST PRACTICES FOR SERVICE PROVIDER BILLING EMC Avamar and EMC Data Protection Advisor • Multi-tiered billing accurately reflects data processed • Automation streamlines reporting • Deduplication reduces redundant data EMC Solutions Group Abstract This white paper discusses billing best practice recommendations for EMC® Avamar® based Backup-as-a-Service (BaaS) solutions. The white paper details the benefits and design considerations of different BaaS billing solutions, and presents a new multi-tiered billing approach, and includes a review of how to implement these billing solutions. August 2012
  • 2. Copyright © 2012 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All trademarks used herein are the property of their respective owners. Part Number: H10789 EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 2 EMC Avamar and EMC Data Protection Advisor
  • 3. Table of contents Executive summary ............................................................................................................................. 5 Overview .......................................................................................................................................... 5 Business case .................................................................................................................................. 5 Solution overview ............................................................................................................................ 5 Key results/ recommendations ........................................................................................................ 6 Introduction ....................................................................................................................................... 7 Purpose ........................................................................................................................................... 7 Scope .............................................................................................................................................. 7 Audience.......................................................................................................................................... 7 Terminology ..................................................................................................................................... 7 Backup-as-a-Service billing options.................................................................................................... 9 Baseline billing recommendations ................................................................................................... 9 Billing models .................................................................................................................................. 9 Single-tiered primary protected data report ................................................................................. 9 Back-end deduplicated capacity................................................................................................ 10 Multi-tiered primary protected data report ................................................................................. 10 Premium uplifts/billing options ..................................................................................................... 13 Billing implementation ..................................................................................................................... 14 Billing implementation ................................................................................................................... 14 Avamar built-in reports .............................................................................................................. 14 Avamar Postgres and MCS databases ........................................................................................ 14 Data Protection Advisor ............................................................................................................. 14 Avamar Management Console Command Line Interface............................................................. 14 Primary protected data reports .................................................................................................. 15 Back-end Capacity report .......................................................................................................... 16 DPN Summary report ................................................................................................................. 16 Multi-tiered primary protected data report ................................................................................. 18 Automating reporting ....................................................................................................................... 19 Automating reporting ..................................................................................................................... 19 Using Avamar reports ..................................................................................................................... 19 Using DPA ...................................................................................................................................... 19 Creating reports through ODBC ...................................................................................................... 19 Conclusion ....................................................................................................................................... 20 Summary ....................................................................................................................................... 20 Findings ......................................................................................................................................... 20 EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 3 EMC Avamar and EMC Data Protection Advisor
  • 4. References ....................................................................................................................................... 21 White papers ................................................................................................................................. 21 Product documentation.................................................................................................................. 21 EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 4 EMC Avamar and EMC Data Protection Advisor
  • 5. Executive summary Overview A service provider’s ability to successfully and competitively monetize an “as-a- service” offering in the marketplace is one of the most critical aspects to its operational success. The EMC® Avamar® backup solution is an enterprise backup solution that is popular in the Backup-as-a-Service (BaaS) market. One of the reasons for Avamar’s popularity is its deduplication efficiency. Several options are: • Service provider charge each customer for deduplicated space utilization. • Customers are charged based on non-deduplicated space utilization. • Customer are charged based on the space being processed (also known as protected) on their systems. Service providers are using each of these options operationally. Which is the correct approach for your business model? While this white paper cannot answer that question, it outlines a variety of potential billing models that you can consider. The discussion of each model includes its benefits and drawbacks along with a discussion about how you can implement it. Business case Service providers that develop and deliver as-a-service solutions have a common requirement for the billing processes. There must be an effective way to track, report, and bill their customers based on the usage rates agreed to in the contracts. When it comes to “standard” storage components and Storage-as-a-Service (STaaS) offerings, this can be a straightforward process. Monthly bills are based on the amount of raw storage consumed by the user and are determined from the base storage subsystem. When you view BaaS, a key driver in cost reductions for the service provider is to use solutions that employ data deduplication. This reduces the amount of data that needs to be retained and stored by the system. This enables service providers to dramatically reduce the amount of storage infrastructure needed to support the backup processes yet still provide the same levels of recovery capabilities, which yields significant cost savings. When you create a billing process, it is no longer a simple one-to-one ratio of data to storage, because of the deduplication technology used during the backup processes. Multiple backup agents may reference the same data objects across datasets, which makes it difficult to count the effective used storage. Different types of data deduplicate at different rates, some higher and some lower. When you couple these characteristics with the retention policies, you can add a lot of variability into the storage usage profile. Solution overview Within the EMC data protection solutions, there are different billing methods that can be supported based on the service provider’s needs, including: • Protected front-end primary data • Back-end deduplicated data • The types of data being stored and their deduplication efficiency EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 5 EMC Avamar and EMC Data Protection Advisor
  • 6. Some combination of these methods Different cost tiers can be created for different types of data based on: • How well data deduplicates • Retention periods • The amount of data a client sends to the service provider during a backup session The service provider must decide which model to use. It is important to understand the benefits and risks for each billing model so you can plan the cost structures appropriately to drive a profitable BaaS business. Key results/ This solution provides several multi-tiered billing models for service providers that recommendations can: • Provide a more effective way to manage billing for customers • Reduce backup costs through different types of billing options • Simplify reporting by automating the process EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 6 EMC Avamar and EMC Data Protection Advisor
  • 7. Introduction Purpose This white paper provides a detailed discussion of the different billing models and corresponding implementation options a service provider must consider when building a BaaS solution. Scope The scope of this white paper is to: • Show how a service provider can build a billing model for structured and unstructured data • Provide overviews of typical billing models and introduce additional multi- tiered models • Use Avamar’s automation mechanisms to generate billing data Audience This white paper targets the business and technical architects of service providers s, who are responsible for developing and implementing the billing aspects of a BaaS solution. It is expected that the reader will have baseline knowledge of Avamar, EMC Data Protection Advisor (DPA), and the terms associated with disk-based deduplication techniques. Terminology This white paper includes the following terminology. Table 1. Terminology Term Definition Deduplication Data deduplication is a method of reducing storage needs by eliminating redundant data. Only one unique instance of the data is retained on storage media. Redundant data is replaced with a pointer to the unique data copy. Garbage collection Improves write performance by eliminating the need to perform whole-block erasure before every write. Garbage collection works in the background and accumulates data blocks previously marked for deletion, performs a whole block erasure on each "garbage" block, and returns the reclaimed space for reuse by subsequent write operations. Retention policies The rules that are used to determine how long data is maintained on the system and available for recovery. Service provider A company in the IT industry that provides organizations with communications, storage, processing, or other services. Although the term service provider can refer to organizational subunits, it is generally used to refer to third-party or outsourced suppliers. Structured data Data that is identifiable because it is organized in a structure. The most common form of structured data or structured data records (SDR) is a database where specific information is stored, based on a method of columns and rows. Tenant A tenant is one of potentially many unrelated customers that are being serviced by the service provider’s shared infrastructure. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 7 EMC Avamar and EMC Data Protection Advisor
  • 8. Term Definition Unstructured data Data that has no identifiable structure. For example, images, videos, email, documents, and text are all considered unstructured data within a dataset. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 8 EMC Avamar and EMC Data Protection Advisor
  • 9. Backup-as-a-Service billing options Baseline billing It is important that you create the billing structure with a base cost derived for a recommendations particular base type of data with a specified base retention policy. This allows a low- cost charge for customers who do not need advanced data capabilities, but provides uplifts to cover the costs for customers who do require advanced data capabilities. As outlined in this chapter, these premium uplifts can be structured separately. Often, service providers want to use a simple single-price or primary-GB costing model for their customers. What this means is that when a customer backs up data, regardless of the data type in the dataset, the customer is charged at a common rate based on a fixed retention policy. When creating the costing structure, the service provider must make assumptions about the amount of data that will be unstructured (such as data that deduplicates well) versus the amount of data that will be structured (databases or highly unique file types such as JPGs that do not deduplicate well). If the data mix turns out to contain more structured data than what was assumed, the monthly price point will not deliver positive profits. Alternatively, if the data mix is more unstructured than planned, the data mix will return a better profit to the service provider. Therefore, a single price tier for all types of primary source data is risky. Before delving into the various use cases for Avamar billing implementations, the following is a breakdown of what Avamar BaaS service providers are using: • Ninety-five percent use Avamar’s Bytes Protected report (also known as Primary Protected report). • Four-point-nine percent use a combination of Avamar’s Bytes Protected report with Avamar’s v_dpnsummary mod_sent value. This specifies how many bytes were transmitted from a backup client to the Avamar grid for a specific backup job. • Zero-point-one percent use Avamar’s Back-End Capacity report (this feature was added in Avamar 5.1). Billing models This white paper outlines the following billing models: • Single-tier primary protected data report • Back-end deduplicated capacity • Multi-tiered primary protect data report (two- and three-tiered) Single-tiered primary protected data report This is the most popular billing mechanism implemented by service providers for Avamar-based BaaS solutions. Avamar incorporates several mechanisms to report on the amount of data being protected. In addition to being discussed in this white paper, this report is detailed in the Reporting section of the EMC Avamar 6.1 Administration Guide. There are several variations of the Bytes Protected report, which enable this report to run on a system-by-system basis with or without a date range. Each of these reports will be discussed in further detail in subsequent sections of this document. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 9 EMC Avamar and EMC Data Protection Advisor
  • 10. You can report on primary protected data with Avamar alone or with Avamar and DPA. Avamar standard reports provide basic primary-capacity reports and you can use them as a starting point to develop custom SQL queries. This may be necessary because the basic reports do not include the Avamar domain to which a particular client belongs. Once the report is compiled, DPA can also provide this information, without the use of custom SQL queries, and can be used to upload the data to a billing system. Back-end deduplicated capacity The best way to create a billing process is to base it on the amount of data being stored on the back end by each customer. Because we are using deduplicated storage, there are going to be data objects shared across customers. This means that you cannot simply account for each used byte of data. More accurately, you have to count each used byte for each customer; otherwise, one customer is paying for shared data objects from which other customers would benefit. Within the Avamar solution, we used the Back-end Capacity report to determine the amount of deduplicated storage used on the system by each customer. This takes into account data deduplication within all the customers’ agents being protected by the Avamar system. When you total the amount of data represented by the unique data objects, you can provide the output of consumed deduplicated bytes. Because different types of data deduplicate at different rates, you will not know how much back-end storage is going to be used as a sole function of protected primary data. This is not an issue with the Back-end Capacity report, as each customer is charged based on back-end utilization. In addition, this report counts all the data kept on the system for retention purposes. By summing up all deduplicated storage along with the storage that is associated with retention, you can determine an individual customer’s exact Avamar storage use. While customers are aware that deduplication percentages vary greatly based on file type, they cannot accurately detail what types of files they have. Because of this, it is difficult for a service provider to gauge, during the sales process, how much storage space a customer’s data will consume and be billed for until the end of the first month’s billing cycle. The Back-end Capacity report is not a simple real-time SQL query of the Avamar database. The Back-end Capacity report involves a multi-step process that simulates a selective data replication for each set of customer servers. Running this report can take a significant amount of time and depends on the number of servers and retention policies. Because this report details deduplicated storage for a specified set of clients, it is best run on a per tenant basis. Note You cannot use DPA information in the Back-end Capacity report. Multi-tiered primary protected data report Because of the risks of having a single price tier for all data types, a better alternative is to create a tiered pricing structure for different types of data. In general, a three-tier model makes the most logical sense, but you can also use a two-tier model. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 10 EMC Avamar and EMC Data Protection Advisor
  • 11. The multi-tiered models have the following attributes: • Two-tiered  Structured data  Unstructured data • Three-tiered  Structured data  Unstructured data without exclusion  Unstructured data with exclusion The fundamental goal of these multi-tiered reports is to provide service providers with the ability to separate unstructured and structured data. For these reports, the unstructured and structured data will be determined by the type of Avamar agent used to perform the backup. All OS agents back up unstructured data, while agents for Microsoft SQL Server, Microsoft Exchange, Oracle, DB2, Sybase, and so on, back up structured data. A service provider must carefully consider the average change rates for structured and unstructured data. The typical change rate for an OS agent client is three-tenths of one percent per day, while the average change rate for a database is typically six percent. This is a large difference, which has a significant impact on: • Backup windows • The amount of data required for retention windows • The amount of network traffic a client consumes When you compare the single-tier primary protected report against the two- and three-tiered reports, the previous change-rate discussion becomes critical and justifies why a multi-tiered reporting mechanism may be the best solution. The difference between the two- and three-tiered reports provides a way to implement exclusion lists into the backups. You can use the exclusion lists to separate data types that: • Incorporate compression, encryption, or other techniques • Eliminate white space within the file • Do not deduplicate or compress well By providing a way to bill separately, based on these datafile types, you can better manage the cost per GB. Common file types for unstructured data includes 7Z, JPG, AVI, MOV, TGZ, and so on. Unstructured data that deduplicates well and typically would not be in the exclusion list, includes file types such as DOCX, PDF, DOC, PPT, PPTX, and so on. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 11 EMC Avamar and EMC Data Protection Advisor
  • 12. By using multi-tiered data reporting, you can specify different billing rates for the different data types based on deduplication efficacy. By charging directly based on usage for the different data types, the costs charged will cover the appropriate costing structures. Reporting for this type of billing structure requires custom SQL scripting through Avamar’s Open Database Connectivity (ODBC) interface. This enables you to group the data based on the available fields maintained in the Avamar database. Combining this data provides the information required by the service provider’s billing infrastructure to generate the appropriate invoices and display usage reports for the different tenants. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 12 EMC Avamar and EMC Data Protection Advisor
  • 13. Premium When delivering backup services, some customers require expanded capabilities to uplifts/billing support their business needs. The costs for premium services should be billed options separately or as a different type of Service Level Agreement (SLA) only to those customers requiring the services. Table 2 describes the available premium services. Table 2. Premium services Service Description Data restores Many service providers provide a specific number of data restores during the monthly billing period. Restores above that level incur a cost or restoration fee. This maintains control over customers who misuse the backup system to provide disaster recovery services by performing daily server restores to an alternate location. Data output to tape Using extended retention, through a media access node, data can be written to tape on a defined, periodic basis (generally only on a monthly basis), which provides longer-term retentions on lower-cost media. The cost to write to tape is based on price/GB written, and can be tiered based on the data amounts that are written. Longer retention policies The service provider can increase the price/GB or percentage uplift of the base price/GB charge to cover a longer retention policy on a disk. Typical Avamar BaaS deployments use a 30- day retention period. You must consider the implications of longer retention periods in storage space utilization. EMC recommends you assume a three-tenths of one percent daily change rate for file server type workloads and a six percent daily change rate for application-specific backups. Replication If customers want their data replicated offsite, there is an additional price/GB to cover this service. The service provider can offer this type of service in a variety of ways. Replicating a customer’s Avamar system to a service provider’s Avamar system in a different geographic area has recently been generating interest. Another alternative is to provide customers with the ability to replicate their data from one of the service provider’s data centers to another data center located in a different geographic area. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 13 EMC Avamar and EMC Data Protection Advisor
  • 14. Billing implementation Billing Depending on your environment and operational requirements, you have several implementation options for generating the Avamar billing reports discussed in the previous chapter. You can use Avamar to generate billing-related information based on your requirements. Avamar built-in reports Avamar provides many built-in reports that you can generate and modify directly from the Avamar Management Console. You can customize these reports to create and gain access to customer-specific data. It is also possible to use DPA to provide additional reporting capabilities for Avamar. Despite the capabilities of Avamar and DPA, neither provides the ability to generate new reports that combine information from existing Avamar reports. Avamar Postgres and MCS databases Avamar uses a Postgres database to track of all its activities. The Management Console Server (MCS) datafile name is mcdb and is located on the utility node in the /usr/local/Avamar/var/mc/server_data/postgres folder. The MCS database provides read-only access through defined public views and tables. You can access the information in this database using custom SQL calls through ODBC. You can create highly customized reports when you access mcdb information. Avamar provides several MCS database views. These public views are documented in Appendix C of the EMC Avamar 6.1 Administration Guide. You can access some views through the Avamar management GUI. For example, the Backup Statistics screen provides statistics by using data from the v_activities_2 database view. The v_reports view lists all the MCS database views, along with the SQL calls used to generate them. You can find additional information about this view in Appendix C of the EMC Avamar 6.1 Administration Guide. We used this view into the mcdb to generate the customer queries needed to create several reports discussed in this white paper. Data Protection Advisor DPA provides a mechanism to automate the collection, analysis, and reporting of Avamar grid usage by tenant, and can be used to customize many of the reports that Avamar provides, without creating custom SQL queries. Avamar Management Console Command Line Interface The Avamar Management Console Command Line Interface (MCCLI) provides an essential capability to the successful integration of Avamar into a service provider’s portal. During the development of our reference portal and BaaS implementation, we used the MCCLI extensively. Note MCCLI is not built as a reporting tool and has no mechanism to create, edit, or run Avamar reports. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 14 EMC Avamar and EMC Data Protection Advisor
  • 15. Previous sections of this document have discussed the various types of reports that are either used by service providers or that EMC recommends while designing a BaaS solution. Each of those reports, in addition to one hybrid, is described in this section. Primary protected data reports Billing the customer based on the amount of data protected by the BaaS solution is the most popular billing mechanism in production. For more information about this billing method see Chapter 8 of the EMC Avamar 6.1 Administration Guide. There are several variations of the Bytes Protected reports, which enable these reports to run on a system-by-system basis with or without a date range. These reports provide the maximum number of bytes protected for each customer. The Activities – Bytes Protected Client report provides the maximum number of bytes protected by the Avamar system in the last two weeks for each client. Alternatively, the Activities – Bytes Protected Client – 2 report specifies the same information for activity that occurred during a specified date range. You can run either report on a daily basis or once per month. Service providers who run the report on a daily basis typically do so to: • Provide daily updates to the self-service portal • Use the data to obtain the maximum number of bytes protected during a 30-day period, for which they bill the customer Recommendations EMC recommends that you run the Activities – Bytes protected Client – 2 report for the preceding month on a daily basis, and update the self-service portal based on the results of that report. EMC also recommends, though not strictly a billing issue, that you display the output of the Activities – No Activities and Activities – No Checkins information on the self- service portal. This ensures that the customer is aware of systems that have not been backed up recently. These two reports do not include any storage used on the Avamar grid for retention purposes. EMC recommends that you build a rate that includes the length of retention for the Avamar infrastructure into the price per GB protected. If you are going to bill a tenant based on the quantity of primary protected storage, EMC recommends that you bill the customer using one of the following models: • Run the Activities – Bytes Protected 2 report for the preceding month to obtain the largest amount of data protected on each system. • Bill based on an average of the Activities – Bytes Protected report run on a daily basis for each system. If run at the root domain, the output of the Bytes protected Client and the Bytes Protected Client 2 reports list each server that was backed up and the corresponding amount of storage protected. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 15 EMC Avamar and EMC Data Protection Advisor
  • 16. Creating a per-tenant (Avamar domain) report requires additional work because the report lists the servers, not the Avamar domains, to which they belong. To collect this data on a per-tenant basis: • Use a report template (for example, Clients Report template) that includes the client domain in the report. • Use the Activities Report template that states that the client’s domain can be included. • Customize the report to include each tenant instead of the root domain. • From the report, extract the client name and, using Postgres/ODBC calls, map the name to its domain. You can use a view such as v_group_members, which includes client_name and full_client_name. Back-end Capacity report The Back-end Capacity report provides information on the deduplicated storage usage by systems or tenants. The Back-end Capacity report performs a simulated replication operation of an empty Avamar grid for the specified systems or tenants. From this report, you can determine the exact Avamar grid usage and include any deduplication benefits of the specified targets. This information includes all retention data and any usage of globally deduplicated items. Some service providers script the Back-end Capacity report to run once a month for all their tenants. By scripting the Back-end Capacity report to run each tenant sequentially, the variability in how long each run takes is irrelevant. While we feel this is appropriate for small- and medium-sized Avamar BaaS deployments, this solution is not yet highly scalable and should not be deployed in large BaaS environments with hundreds of tenants. Notes • With Avamar 6.0 and 6.1, you cannot use this report to determine the back-end capacity used by a system being backed up to an EMC Data Domain® system. • Only one instance of this report can be run at a time, requiring individual runs for each domain or client grouping. Only one instance of the Back-end Capacity report can be run simultaneously, because the system has to parse through all its metadata to determine how much of the back-end capacity is being used by the targeted systems. DPN Summary report You can use the DPN Summary report for billing through Avamar’s MCS report, Activities – DPN Summary, or by directly accessing the v_dpnsummary MCS database view through ODBC calls. You can use the information available from the DPN Summary report by itself or combined with other report data, such as the Primary Protected report. Table 3 describes the data available in the DPN Summary report. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 16 EMC Avamar and EMC Data Protection Advisor
  • 17. Table 3. DPN summary report Variable name Description Host The client hostname Mod_sent The number of bytes of files that were modified since the last backup and sent to the Avamar grid for protection Modnotsent The number of bytes of files that were modified since the last backup but were not sent to the Avamar grid due to global deduplication Totalbytes Number of bytes processed Note You can use the data in this report to generate the Primary Protected dataset size on a per-host basis. By running the DPN Summary report for a specified monthly billing period and summarizing mod_sent, you can calculate the amount of data a client has sent to the Avamar grid. However, this number does not include deduplication. To collect deduplication information, the modnotsent variable needs to be used so that it is possible to distinguish between what Avamar is protecting and what is captured in the primary protected reports. The report does not consider data that has been expired on the Avamar grid because of the report’s retention policies. In addition, the report does not consider the amount of Avamar space being used for data retention. Example As an example of the challenges inherent with using the mod_sent method for billing, consider the use case of a BaaS environment in which a customer backs up a 1 TB database. Assuming reasonable deduplication efficiency, the Avamar client sends 300 GB to the Avamar grid for the initial seeding of this file on day one. You can also assume that data retention is set to 30 days and that there is a three percent day-to- day change rate on the file. Once the initial seeding of 300 GB is complete, only 3 GB is sent on a daily basis. You have a total mod_sent of 387 GB = 300 GB + (29 * 3 GB) for the first month. That also equals the usage of the grid from a retention period, which is reasonable. The concern for the service provider occurs in month two, where only the change-rate data is sent. The mod_sent for month two is 90 GB = (30 * 3 GB). The customer is satisfied, while the service provider is not, as the total usage of the backed Avamar grid, at the end of month two is 387 GB = 300 GB + (29 * 3 GB). Note There is more post processing required of DPN summary reports as they report on a per-host basis. The tenant needs to summarize this before uploading to the billing system. Custom reports You may want to generate custom reports based on a combination of primary protected and mod_sent data. One typical use case that tracks mod_sent is to use that information to validate what the daily change rate is for a particular customer. You can do this by tracking mod_sent in combination with totalbytes and modnotsent. Taken together, you can use totalbytes and modnotsent to determine EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 17 EMC Avamar and EMC Data Protection Advisor
  • 18. the daily change rate for systems and tenants. By having this change rate data, you can fine tune the billing assumptions around the quantity of storage used for data retention. Multi-tiered primary protected data report EMC recommends a custom multi-tiered billing model as an alternative to a higher- risk single-price approach to billing. The recommended tiers are: • Unstructured data with exclusions of specific, low deduplicatable data types, whose typical deduplication efficiencies are relatively low • Unstructured data with no exclusions types, whose typical deduplication efficiencies are typically very high • Structured datasets There are no existing reports that collect this information, so you must create a custom report using ODBC access to the Avamar database (MCDB), then using the database views. You must collate and collect information from individual backup jobs, including: • The type of agent plug-ins used for the backups (OS agents versus application agents) • The size of the file • Dataset names (identify which datasets have exclusions versus no exclusions) • Customer domain as set up within Avamar By collecting this information for each back-up job, you can determine: • The amount of data, by data type, that was sent to the Avamar grid • The type of agent plug-in used • The customer domain You can then group the data, according to typical deduplication efficiency and plug-in used, to be billed back to the customer. Custom SQL scripting through Avamar’s ODBC interface is required to collate and organize the information. Once collected, the information is forwarded to your billing infrastructure to generate the appropriate invoices and display usage reports for the different tenants. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 18 EMC Avamar and EMC Data Protection Advisor
  • 19. Automating reporting Automating In order for you to cost-effectively scale out an as-a-service offering, automation is a reporting key requirement. The recommendations are not specific to any of the reports previously listed; they are a general overview of what automation options are available. Using Avamar You can create Avamar reports using the Avamar Admin > Tools > Manage Reports reports dialog box, and specify the attributes desired. You can create this report at the root domain to include all Avamar domains or a separate instance of this report for each Avamar domain. Your environment and the amount of post processing you wish to perform will dictate which method to use. Creating a mechanism within Avamar to automate the execution of a report requires that you use a custom event profile. For more information, refer to the sections “Creating a customer event profile” and “Events, Notifications, and Profiles” in the Avamar Admin Guide. As part of the event profile creation process, you select a triggering event code. EMC recommends that you use an event code such as “garbage collection initiated” or “garbage collection completed” as the trigger. The automated mechanism requires that the reports be emailed. EMC recommends that you email the reports to a service-provider-controlled mailbox that saves the attachments to a local file, then you can process the reports as required. Using DPA You can create the same reports using DPA. DPA provides additional reporting capabilities without customer scripts. It also provides more capabilities for automating report generation than Avamar. Creating reports The most powerful and flexible automation option is to access the data from through ODBC Avamar’s database directly. You create the report through ODBC/SQL calls against the Avamar Postgres database. Appendix C of the EMC Avamar 6.1 Administration Guide provides a list of precreated views into the Avamar database. Avamar uses many of these views for the prepackaged Avamar reports. For example, the v_dpnsummary MCS database view includes the underlying data used to create the Activities – Bytes Protected Client reports. The critical data is host, activity, and totalbytes. Totalbytes provides the number of bytes processed on the client. EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 19 EMC Avamar and EMC Data Protection Advisor
  • 20. Conclusion Summary There must be an effective way to track, report, and bill service providers’ customers based on the usage rates agreed to in their contracts. Different types of data deduplicate at different rates, some higher and some lower. When you couple these characteristics with the retention policies, you can add a lot of variability into the storage usage profile. The financial model and reports used by the service provider must accurately capture and bill customers based on their usage. This white paper provided an overview of several multi-tiered billing models for use with Avamar-based BaaS solutions. These multi-tiered reports provide service providers with a flexible set of billing alternatives that can be customizes for their specific environment. Findings This solution provides several multi-tiered billing models that can: • Streamline data backup and restoration by reducing wasted storage • Provide a more effective way to manage billing for customers • Simplify reporting by automating the process EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 20 EMC Avamar and EMC Data Protection Advisor
  • 21. References White papers For more information, see the following white papers: • EMC Backup-as-a-Service: EMC Avamar, EMC Data Protection Advisor and EMC Homebase • EMC Avamar Technical Deployment Considerations for Service Providers • EMC Avamar Business Deployment Considerations for Service Providers • Creating “Backup as a Service” (BaaS) Solutions Leveraging EMC Avamar Product For more information, see the following product documents: documentation • EMC Avamar 6.1 Administration Guide EMC Backup-as-a-Service: Applied Best Practices for Service Provider Billing 21 EMC Avamar and EMC Data Protection Advisor