SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver

W I N T E R C O R P O R A T I O N
W H I T E PA P E R
SAP BusinessObjects
Planning and Consolidation,
Version for SAP NetWeaver
Performance and Scalability

en t Experts
g em
a
an
M
a ta
D
e
al
Sc
e
rg
La
The

SPONSORED
RESEARCH
PROGRAM

W I N T E R C O R P O R A T I O N

SAP BusinessObjects
Planning and Consolidation,
Version for SAP NetWeaver
Performance and Scalability

March 2010

245 First Street, Suite 1800
Cambridge MA 02145
617-695-1800

visit us at www.wintercorp.com

©2010 Winter Corporation, Cambridge, MA. All rights reserved.

SAP BusinessObjects Planning and Consolidation, Version for SAP NetWeaver: Performance and Scalability 3

A WINTER CORPOR ATION WHITE PAPER

Executive Summary
The use of enterprise performance management (EPM) software for
speeding an organization’s financial reporting and planning and budgeting can have a direct
impact on the bottom line. Closing the books rapidly means that business decision makers at
every level can immediately react to changes in the business environment as revealed by the
financial results. Faster reaction can mean both more leverage from new opportunities and less
money wasted on initiatives that don’t perform. Faster planning, and budgeting according to a
“what-if” plan allows deeper investigation of alternative strategies, with a greater chance that
the final strategy will be effective. Finally, today’s business environment often couples a complex,
rapidly growing organization with a need for deeper financial analysis. This means that planning,
budgeting, and consolidation software must perform and scale better with each version.
SAP BusinessObjects™ Planning and Consolidation software, presently used at an SAP-estimated
1800 customers worldwide, aims to meet the needs of medium- and large-scale organizations
for high-performance, scalable planning, budgeting, forecasting, and financial consolidations.
The SAP BusinessObjects Planning and Consolidation, application version for SAP NetWeaver was
introduced into the market in late 2008.
To verify that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver meets the
needs of likely end users for performance and scalability, WinterCorp reviewed and monitored
testing performed by SAP on a typical configuration. The testing involved a simulated high-
workload use of SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver for
planning, consolidation, and “combined” (simultaneous planning and consolidation activities)
scenarios, using HP/Mercury LoadRunner for the test. Tests were performed for 50, 250, and 500
concurrent users.
Highlights of our findings include:
• The tests that SAP performed were realistic, as to the number of concurrent users, hardware
and software used, and the user scenario for each activity.
• The tests established to our satisfaction that the configuration specified exhibited better-than-
linear scalability in processor consumption as the number of concurrent users increased from
50 to 500.
• Even allowing for delays as users work offline during the processes, it is possible for some
organizations to complete a full consolidation process or “what-if” scenario with participation
by lines of business, intermediary mid-level managers, and senior corporate finance staff—within
a day.
• Scalability from 50 to 500 concurrent users in key subprocesses was better-than-linear on average
in all scenarios.
• Maximum response time for these subprocesses was within a reasonable distance of average
response time. SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver
demonstrated a high degree of predictability in its performance.
• Average response times, in all cases where the end user awaits a response (e.g., reports and
currency conversions), appeared satisfactory to excellent even in the 500-user case.
WinterCorp concludes that, when correctly configured, the SAP BusinessObjects Planning and
Consolidation, version for SAP NetWeaver product should produce satisfactory to excellent performance,
and excellent scalability, in typical business uses involving up to 500 concurrent users.

Copyright © 2010, WINTER CORPORATION, Cambridge, MA. All rights reserved.



Acknowledgements
SAP used IBM System x3950 M2 and x3850 M2 servers to achieve its performance goals for this
project. IBM provided hardware, system software and support for the testing effort. The
configurations used in this testing included eight, six, and two processor configurations.
Hyde Park Partners, LLC contributed to the design of the planning application using expertise
gained through their onsite engagements and supported the Consolidation application effort.




Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Table of Contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1 Introduction & Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Benchmark Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1 Overview of SAP BusinessObjects Planning and Consolidation,
Version for SAP NetWeaver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 The System Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 Scenarios Tested . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Findings – Resource Scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Findings - Response Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Analysis of Test Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

5 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11




1 Introduction
Today’s business environment is exceptionally cost-constrained and risky, with rapid and
unexpected external developments that often affect the bottom line of the business, such as
changes in bank lending behavior. Reacting to these changes in the context of a business plan
requires rapid generation of results, quick re-planning and re-budgeting, and coordinated, swift
implementation of the new budget across highly complex organizations.
Ground zero for this effort is consolidation, planning, and budgeting software—an application
of EPM software. Vendors such as SAP that offer this software are held to a higher standard:
they must improve performance and scalability with each release, to meet the new demands;
and they must handle all of the complexities of organizations with tens and hundreds of distinct
lines of business. The aim of WinterCorp’s benchmark assessment is to see whether SAP
BusinessObjects Planning and Consolidation, version for SAP NetWeaver is able to handle typical large/
complex-organization consolidation/planning needs in the real world.

M E TH ODO LO GY
This WinterCorp White Paper reviews the performance and scalability of mixed workloads
in the SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver architecture.
In developing this paper, WinterCorp operated as an independent industry expert,
reviewing product documentation and critically reviewing test design, test process and
benchmarking measurements in order to arrive at the descriptions and conclusions
presented here. SAP was provided an opportunity to comment on the paper with respect
to facts. WinterCorp has final editorial control over the content of this publication. This
White Paper was sponsored by SAP.
SAP completed the tests described below during the period of December 2009–January
2010. SAP then supplied WinterCorp with the raw LoadRunner statistics for the tests, as
well as summary statistics generated from the tests. In preparing this report, WinterCorp
reviewed the flow diagrams for the scenarios described below as well as the statistics
generated by LoadRunner and SAP, and generated summary statistics that form the basis
for our conclusions in this report.

2 Benchmark Overview
2 .1 OVE RVIE W O F SAP Business O bjects Planning and Consolidation,
version for SAP Net Weaver
SAP BusinessObjects Planning and Consolidation is software that enables organizational planning
and consolidation processes. The aim of the product is to improve planning/consolidation
frequency, accuracy, and effectiveness; reduce budget-cycle times and costs; and increase user
confidence in the validity of financial results. Notable features include support for “self-service”
financial reporting, intercompany reconciliation, legal consolidation, and collaboration. SAP says
that 1800 customers use the application.
The version of the application being tested is SAP BusinessObjects Planning and Consolidation, version
for SAP NetWeaver. This version is available alongside the existing version for the Microsoft
platform. It is built directly on the SAP NetWeaver technology platform (BW, ABAP Application




Server). Whereas in the version for the Microsoft platform, Microsoft SQL Server was the only
database supported, the version for SAP NetWeaver works with any relational database that SAP
NetWeaver supports. The architecture is a typical Web setup (see Figure 1 below), with remote
clients issuing transactions accessing an OLAP-type data store including cubes via .Net Server,
SAP ABAP Application Server, and the NetWeaver BW OLAP Engine.
Figure 1: SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver Architecture

Source: SAP

2 . 2 THE SYSTE M CO NFIGUR ATIO N
The configuration of the test environment is 3 IBM servers for the .NET tier, and 3 IBM servers
for the ABAP/NetWeaver tier. Remote clients are represented by processes generated by
LoadRunner on the .NET tier. The .NET tier runs Windows 2003 Enterprise Edition, 32-bit
version, and the ABAP/NetWeaver tier runs Windows 2003 Enterprise Edition, 64-bit version.
Each .NET server feeds one ABAP server. One ABAP server runs a “central instance” of the ABAP/
NetWeaver software and the Oracle database, with the other two running “dialog instances”
that connect to the central instance.
The test environment uses between 1.5 and 19 million records of generated data per test and 55
to 151 million records of total data. Some of this data is pre-formatted as cubes, and then pre-
loaded into the Oracle data store; the rest is “entered” during the test as “user input.”
Likely areas of stress in the configuration during a test run include “write back” of data input
during the test, query-type reads for reporting, and Data Manager that will handle data loading
and logic execution.
The test configuration includes SAP NetWeaver Business Warehouse (BW). An optional component
of the product to improve performance is SAP NetWeaver Business Warehouse Accelerator (BWA).
The test configuration did not include this component.




2 . 3 S CE NARIOS TESTED
SAP identified, modeled, and tested planning, consolidation, and combined scenarios. Combined,
these scenarios involved more than 450 distinct transactions, or “unit operations.” Each scenario’s
process included multiple “sub-processes” (e.g., Gross Margin calculation) as shown in Figures
2 and 3 below. Each sub-process represented a sequential series of unit operations (e.g., Report
Actuals) that are carried out by a particular type of user (entity-level manager, mid-level manager,
or senior finance person). Sub-processes can occur concurrently or sequentially.
Each scenario was tested using 50, 250, and 500 simulated concurrent users. These numbers of
concurrent users approximated an organization with up to 1200 lines of business that would feed
their own planning and budgeting estimates or their results over the most recent time period into
the system, have these estimates/results reconciled by mid-level managers, and have the results
consolidated for the reporting use of senior finance personnel at corporate headquarters.
The tests make a distinction between end users, concurrent users, and active users. For the
purposes of this report, we define end users as all of the people using the system at one point
or another in a scenario, although all may not be using the system at a given point in time.
Concurrent users are those end users who are using the system at a given point in time. Active
users are those who are not only using the system at a given point in time, but are actively running
processes within the system, simultaneously. The tests measure scenarios in which 50 (or 250,
or 500) concurrent users use the system during each sub-process, although not all are active at
any one time.
An overall flow diagram for the planning scenario is shown in Figure 2. The scenario simulated
Gross Margin and Human Capital Management (HCM) entry and computation in parallel,
followed by overall financial analysis/reporting activities. HCM mid-level manager activities
included head count, benefits, and salary input and computation by entity, period, and employee.
Gross Margin entity-level (e.g., line of business) activities included gross margin unit, unit x
price, unit x cost, revenue by entity/customer, and expense detail input/computation/consolidation.
Gross Margin mid-level manager activities included gross margin comparison and identification
of top customers. The senior finance calculations and reporting included expense and expense
driver computations, income statement reporting, and revenue by entity/customer.
Figure 2: Planning Scenario Process Flow

Source: SAP
An overall flow diagram for the consolidation scenario is shown in Figure 3. The scenario simulated
entry of results by entity-level personnel, followed by reconciliation by mid-level managers and
consolidation and analysis/report generation by senior finance. Entity-level inputs were entered
grouped by asset/liability type, and end users also input/computed changes in liability, tax




statements, and validation. Mid-level manager activities included generation and validation of
per-entity statements (balance sheet, income statement, cash flow statement, and equity statement).
Senior finance calculations and reporting included consolidation of each set of per-entity
statements into an organization-wide statement.
Figure 3: Consolidation Scenario Process Flow

Source: SAP
The combined scenario simply split the sets of concurrent users between planning and
consolidation scenarios, and then tested these scenarios in parallel.
In the opinion of WinterCorp, these scenarios represent a reasonable approximation of the core
planning and consolidation activities of a typical mid-sized or large organization with many
entities to include in overall projections or results.

3 Test Results
3.1 FINDINGS – RES OURCE S CALING
Figures 4 and 5 below present the scaling of processor utilization for the planning, consolidation,
and combined scenarios.
Figure 4 shows the average percent of processor utilization at peak load in each scenario, for 50,
250, and 500 users.
Figure 4: Processor Utilization in Each Scenario Figure 5: Processor Utilization Per User In Each Scenario

Source: WinterCorp Source: WinterCorp

Figure 5 shows the average percent of processor utilization per concurrent user in each scenario,
for 50, 250, and 500 users (for easier understanding, the 50-user case is set to value “1”). Figure 5
shows that in each scenario, addition of concurrent users decreases processor utilization per




concurrent user. In other words, SAP BusinessObjects Planning and Consolidation, version for SAP
NetWeaver delivers better than linear scalability of processor utilization for up to 500 concurrent
users in all scenarios.
SAP reports that they have observed much greater database resource requirements during the
Planning scenario than during the Consolidation scenario, and much greater NetWeaver processing
requirements during the Consolidation Scenario than during the Planning scenario.
There appears to be some “headroom” to scale beyond 500 concurrent users in the consolidation
scenario. However, the planning and combined scenarios show peak average utilization rates of
approximately 85% at the 500-user level, indicating that addition of more concurrent users may
call for increased processor capacity.
3. 2 FINDINGS - RES PO N S E TIM E
Of course, less-than-full processor utilization may not translate into flat response time when
SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver is scaled from 50 to 500
concurrent users, if some sub-processes are I/O-bound or communications-bound. To determine
if this is the case, we analyzed response times for unit operations. In particular, we sought to
determine the likely user “wait time” when an end user requested a report, since “endless waits
for a report” are the most frequent cause of user dissatisfaction.

Figure 6: Average Response Times Figure 7: Average Response Times
For Each User Type, Consolidation Scenario For Each User Type, Planning Scenario

Source: WinterCorp Source: WinterCorp

Figures 6 and 7 show the average response times for report requests for each user type (line-of-
business “entity” manager or mid-level manager/senior finance person) in the consolidation and
planning scenarios for 50, 250, and 500 users. Combined scenario results are comparable.
For the planning and consolidation scenarios, minimum response times per report request range
from 1 to 19 seconds, average response times range from 1 to 41 seconds, and maximum response
times range from 1 to 103 seconds. The overall average response time was approximately 5
seconds. These results do not include programmed “think times” (meant to simulate offline work
by the user), or processing time for other parts of a unit operation.
Figures 6 and 7 also show that as the number of users is doubled, report-request response times increase,
on average, by 23%. Any increase of less than 100% indicates good scalability. Similar results hold for
“write-back” and “data manager” transactions, the other two kinds of transactions in a unit operation.
We conclude that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver
demonstrates, on average, better-than-linear scalability to 500 concurrent users.




Figure 8 shows the average standard deviation
Figure 8: Response Time Variation,
of response times for each user type in the 500- Planning and Consolidation Scenarios
user case. This figure can be used to understand
the predictability of response times. Consistency
in response times typically leads to increased
user satisfaction with the product.
Since, 99% of the time, results fall within three
standard deviations of the average, these results
suggest that product users will only see report
requests that take longer than 30 seconds to
run, less than 1% of the time. Of course, your
mileage may vary, depending on the architecture
chosen. In other words, SAP BusinessObjects
Planning and Consolidation, version for SAP Source: WinterCorp
NetWeaver demonstrates a high degree of
predictability in its performance.

4 Analysis of Test Results
The results discussed above indicate that SAP BusinessObjects Planning and Consolidation, ver-
sion for SAP NetWeaver achieves satisfactory to excellent performance and excellent scalability for
realistic planning and consolidation scenarios that involve between 50 and 500 concurrent users.
For situations in which yet faster system response is needed, customers may wish to consider
the BWA option, which aims to accelerate SAP BusinessObjects Planning and Consolidation, version
for SAP NetWeaver performance.
One interesting alternative identified by the test results is what might be called “rolling” budget-
ing. The present planning scenario models a process in which plans are made on the basis of a
full set of “before-consolidation” or “after-consolidation” results, and are not altered “on the fly”
by intermediate results captured during a simultaneous consolidation process. Because of the
speed of consolidation, customers could employ a practice in which mid-level managers are given
some flexibility to alter the plans and perform “what-if” scenarios during the consolidation
process and before senior finance has seen the results, based on variations of these results from
plan. For example, a mid-level manager might adjust a line-of-business budget for the remainder
of the year, on the basis of results reported for the current month. After reconciliation of results,
corporate could then review the new mid-level budgets, adjust the overall budget, and go through
a full follow-on planning process—all before the end of the day. Because this process would not
involve any greater peak loads than the ones modeled here, it appears that customers could
employ SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver in this way and
expect to see similar levels of performance.

5 Conclusions
To meet the planning/budgeting/financials needs of medium- and large-sized organizations, SAP
BusinessObjects Planning and Consolidation, version for SAP NetWeaver customers need to scale cost-
effectively to reconcile hundreds of lines of business, and speedily, to allow “continuous” reporting
and planning in a rapidly-changing business environment. In particular, SAP BusinessObjects Planning
and Consolidation, version for SAP NetWeaver customers need rapid report generation, scaling via




proportional increments in system capacity as the business scales, and the ability to handle “mixed”
workloads involving simultaneous planning and consolidation.
The SAP tests monitored by WinterCorp show that:
• SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver, when properly configured
and running with appropriate hardware capacity, scales approximately linearly with the amount
of processor power used for realistic planning, budgeting, and combined scenarios involving
50 to 500 concurrent users at maximum load.
• Response times showed average better-than-linear scalability, and were quite predictable in all
cases, demonstrating end user report display within 30 seconds of request initiation, better than
99% of the time.
• Performance was adequate to excellent in all cases that we considered – for example, an
average of 8 seconds to wait for report display after request initiation in the planning and
consolidation scenarios.
• Handling “mixed” processes and transaction loads during the combined scenario caused little
or no decrease in performance and scalability—as measured by processor utilization—compared
to other scenarios in all cases. This indicates that SAP BusinessObjects Planning and Consolidation,
version for SAP NetWeaver is tuned effectively for a wide range of customer planning/budgeting/
financial use cases.

WinterCorp concludes that SAP BusinessObjects Planning and Consolidation, version for SAP
NetWeaver, when properly configured and running with appropriate hardware capacity, can
provide scalable business planning and consolidation support for at least 500 concurrent
users in highly complex organizations, supporting “continuous” results reporting and deep
“what-if” analysis for the agile business.


WinterCorp is an independent consulting firm
that specializes in the performance and scalability of
terabyte- and petabyte-scale data management
systems throughout their lifecycle.
Since our inception in 1992, we have architected
many of the world’s largest and most challenging
databases in production today. Our consulting services
help organizations define business-critical database solutions,
select their platforms, engineer their implementations,
and manage their growth to optimize business value.
With decades of experience in large-scale
database implementations and in-depth knowledge
of database products, we deliver unmatched insight
into the issues that impede performance and
the technologies that enable success.

245 First Street, Suite 1800
Cambridge MA 02145
617-695-1800

visit us at www.wintercorp.com

©2010 Winter Corporation, Cambridge, MA. All rights reserved.

Duplication only as authorized in writing by Winter Corporation.

SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver

Recommended

Recommended

More Related Content

Similar to SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver

Similar to SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver (20)

More from IBM India Smarter Computing

More from IBM India Smarter Computing (20)

Recently uploaded

Recently uploaded (20)

SAP Business Objects Planning and Consolidaton, Version for SAP NetWeaver