Dw Quest+Oct+2005

From The Editor’s desk
CONTENTS
Greetings! Here’s another information packed edition of DW Quest!

Ø Business In this edition we present you with an informative paper that gives
Intelligence to direction on process-centric approach to business decision-making. This
Business paper promises to give insight on improving capability of a business to
gain insight and manage performance.
Performance
Management The paper on model management deals with the art of managing various
PAGE 1-6 business models associated with project life cycles. Read on to know why
a project needs models and what are the various models available.
Ø Model Tuning information systems has been a time honored system management
Management activity ever since the first information system was built. The article
PAGE 7-15 presented talks about this concept that helps in enhancing performance.

Ø Datawarehous
Happy Reading and here’s wishing all our readers a very Happy Diwali!
Tuning
PAGE 16-20 Do mail in your articles to Quest_BIDW !

Satyam Computer Services Ltd. (NYSE: “SAY”) is a leading global consulting and IT services
company that offers a wide array of solutions for a range of key verticals and horizontals.
Editorial Board
Business Intelligence and Data Warehousing is a focused competency unit whose mission is to Anita Shelton
enable customers to take better decisions by providing world-class BI solutions. For more Bala Seshadri
information please refer to www.satyam.com. Hrishikesh Gokhale
Disclaimer Manasa Subbaraya
The views expressed in this document are not those of Satyam Computer Services Ltd, and any Rajasekhar Nanduri
errors or omissions in the contents are attributable to the respective contributors / copyright holder. Ruy Fregoso
Customer names are withheld to ensure confidentiality.
© All rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or Vinod K
transmitted in any form or by any means, electronic, mechanical, photocopying, recording or Srikanth V C
otherwise, without the prior permission of the publisher.
CONTACT INFORMATION
Brian C. Coffey Ramakrishnan TS SanjayKumar Joshi Arun VS Sarvesh Kumar
1000 Windward Concourse Suite 540 Satyam City Centre 1, Gatehall Drive 360 Elizabeth Street 6 Cedarwood
Alpharetta, Georgia 1-10-60 to 63, Satyam City Centre Suite 301, Parsippany Level 40, Melbourne Central, Chineham Business park
USA 30005 Begumpet, Hyderabad-500 016 NJ 07054, USA Melbourne, VIC, Australia Basingstoke, Hampshire
(W) 770-442-3111x2201 (W) +91-40-5523 2323 (W) +973-656-0650 (W) +61 3 86605600 England-RG248WD
(M) 404-435-0472 (M) 91-9849830753 (M) 518 248 7887 (M) 614 1616 3136 (W) +44-0-1256-394100
(M) 0044 791 933 9137

Kiran Cavale Sastry Varanasi Victor Samuel Indira Munjuluri Venkat Narayanan
#97, Galaxy Towers, Satyam City Centre #97, Galaxy Towers, 251-252, Solitaire Knowledge Dynamics-A Div of BI
G.N.Chetty Road 1-10-60 to 63, Satyam City Centre G.N.Chetty Road Corporate Park, Mumbai (W) : (65) 6324 6244 x 116
T.Nagar, Chennai-6000 17 Begumpet, Hyderabad-500 016 T.Nagar, Chennai-6000 17 Maharashtra 40 0093, Fax : (65) 6324 4355
(W) 91-44-2826120 (W) +91-40-5523 2323 (W) 91-44-2826120 (W) +91-22-5695 0166 M: (65) 9682 1535 (Singapore)
(M) 9841080505 (M) 91-9849496082 (M) 9841027947 (M) 91-9821723010 M: (91) 9840702328 (India)

Business Intelligence and Data Warehousing Group- Satyam Computer Services Ltd.

Business Intelligence to Business Performance
Management
(Gayatri Jayanthi)

Abstract

In the face of increasing business environment, Business Intelligence alone is in-
sufficient to manage the performance of an enterprise. A new methodology is
required to build, communicate and execute better strategic plans. Business
Performance Management (BPM) is a new term that denotes a process-centric
approach to business decision-making. This is intended to improve the capability of a
business to gain insight and manage its performance at all levels by combining
stakeholders, managers, staff, customers and suppliers within an integrated
management environment. It combines the management methodologies like Six-
Sigma with business requirements for executive reporting and budgeting.

Some of the components that are embedded in Business Performance Management
are

• OLAP – Online Analytical Processing
• Query and Reporting
• ETL
• Analytics
• Budging
• Consolidation
• Metrics
• Portal dashboards
• Scorecards

Challenges faced by Business

1. Attaining Strategic Goals: In any organization, there must be a shared
commitment towards strategic goals. If there is a divergence observed in this,
then there are chances that resources will be mis-utilized. Hence it is BPM
which needs to drive strategic focus towards the same goals by broadcasting
that local goals are congruent to the Corporate Objectives.

2. Optimizing Communication between Organization Units: As the
organization grows in size, it is a huge challenge to have a cross
communication among the various business units and interaction between their
management process. Every unit is focused on their local goals and targets

Business Intelligence and Data Warehousing Group- Satyam Computer Services Ltd. 1

rather than attaining the organizational goals. BPM enables collaborative
management by providing cross departmental data under a single umbrella.

3. Decision Making: Apart from the timely availability of data, the biggest
question that a manager needs to answer is how far the data is accurate and
consistent as the decision made on this data is like working in darkness with
out a clue of the under lying facts.

BPM provides a fact based decision that is based on controlled sources.

4. Knowledge Sharing: It is a known fact that even today many of the
information analysis is done with the help of spread sheets. This information
gets locked with the person who writes it. This is specially observed during
Budgeting and forecasting where the data gets locked in spreadsheets
unavailable in the primary systems. BPM looks forward in evolving a complete
transparency by documenting all forms of management information.

5. Accessing other Operational systems: In addition to all the above
challenges, interfacing other Operational Business systems such as Supply
Chain Management, Enterprise Resource Planning, Employee Relationship
Management and Risk Management is also essential as they play a major in all
executing the strategic plans.

6. Adapting to Business Demands: Responding in a quicker way to market
demands is also seen as a major challenge.

Need for BPM

It allows managers across the extended supply-chain, to collectively measure
performance, anticipate results and drive profitability. It brings together loosely
coupled business units of an organization and moves forward meeting organizational
goals. Performance Management is just one of the management tools which can be
used to measure what your business is doing and also help to align the activities of
people and processes to achieve the desired goals.

• BPM provides an infrastructure to have a peer – to – peer collaboration
between Business Units for facing the changing Business demands with out
losing control over the business.
• Strategies like outsourcing, shared services lead to a new organizational model
which is open and fluid. BPM helps managing the Business Processes by
protecting sensitive information of each participant organization.
• Many huge organizations have deployed Enterprise Business Applications such
as ERP, Supply Chain Management to increase the efficiency of the Business by
streamlining the operational processes and generating valuable data. But many
a times BI on these ERPs focus either on summarized data or provide massive
information which may not be the demand of operational managers. BPM


absorbs the operational data to the granularity that makes sense for the
monitoring and analysis management activities.

This paper will focus on the need for moving towards BPM, challenges in
implementing BPM and also how the increase in BPM implementation would influence
service providing companies.

Introduction

Data on its own does not have a meaning unless it is interpreted to make it
information. Data will turn to be informative when used for analysis or reasoning.
Any organization which empowers itself to utilize its information in its decision-
making and know its strengths and weakness will head towards success. As
organizations realized the importance of information, they turned away from
decisions made on intuition to a fact-based analysis. This lead to the evolution of
Business Intelligence, organizations started automating more systems and hence
more data is available. However collection of all this data in a single repository was a
challenge which was answered by Data warehousing technologies. Business
Intelligence is now the art of extracting large amounts of data, turning it into
information and presenting it in an actionable knowledge.

With the advent of Business Intelligence, organizations felt the demand of aligning
their action with strategy to improve its performance. Thus came BPM as it is
recognized as a way to align and optimize its performance such as sales, marketing,
customer service, IT, compliance, Finance and HR departments across an enterprise.
BPM is seen as the next generation of Business Intelligence (BI). BPM is focused on
business processes such as planning and forecasting. It helps businesses discover
efficient use of their business units, financial, Human, and material resources. BPM
often uses Key Performance Indicators (KPI’s) to indicate the present state of
business and to prescribe the course of action.

According to Gartner "Corporate Performance Management (CPM) or Business
Performance Management is an umbrella term that describes the methodologies,
metrics, processes and systems used to monitor and manage the business
performance of an enterprise." While less than 10% of Global organizations had
implemented Business Performance Management at the end of 2002, Gartner
predicts that more than 40% will do so by 2006.

What is Business Performance Management?

BPM is a combination of 5 components
1. Participants: BPM is about how people come together in order to drive the
performance of the organization. There are many business activities namely
manufacturing products, fulfilling orders etc which must be controlled and


organized by management activities. BPM identifies different roles that any
individual can play with regard to a particular management activity
• Stakeholders
• Suppliers
• Partners
• Influencers
• Managed Staff
• Managers
BPM identifies all these roles and presents a complete view of the management
process.

2. Management Process: This represents the on going dynamic state of the
business. The management teams must interact with their stakeholders,
customers, suppliers and staff as well as other managers in different teams
for developing plans to meet their organizational goals.

3. Management Practices: BPM is a mechanism for recording or discovering
the existing definition, processes, metrics and business rules of the
management. The management practices defines few management activities
like negotiating goals with the stake holders, developing and communicating
plans, variances between prediction and actual and informing Stake Holders.
Apart from this, activity management practices, also define KPI’s that
measure key business drivers.

4. Conceptual Framework: It is a bridge between the management practices
and the under lying systems. BPM defines a universal framework describing
processes, activities, information, business rules and user roles. Without this
Framework it would be extremely difficult to have the same Management
Practices supported by different systems. This is the case as systems are
replaced over time or when different business units are acquired and have
their own systems already in place.

5. Systems and Technology: This is one of the most important component of
BPM. It enables Management Process, Practices and Framework to the
participants of the system. This provides a personalized view as per the role.
There will be a set of tools and applications supporting this underlying system
in order to make the dashboard or scorecards available such as OLAP – Online
Analytical Processing, Query, Reporting, ETL, Analytics, Budgeting,
Consolidation, Metrics, Portal Dashboards and Scorecards etc. The combined
network of all these tools and applications enables the Business managers to
explore innovative alternatives and share them with their colleagues. The
figure below shows the systems, their technology and their inter dependence
to make BPM.


The deployment of Business Performance Management across the enterprise, helps
to communicate with the operational systems, directly or via an intermediate data
warehouse layer.

The figure below shows the pictorial depiction of the components of BPM

BUSINESS MANAGEMENT

PARTICIPANTS

INTERACTIVE MANAGEMENT PROCESS
INSIGHT ACTION

MANAGEMENT PRACTICES

CONCEPTUAL TEXT

SYSTEMS AND TECHNOLOGY


Conclusion

As the business environment becomes highly volatile and fast moving, businesses
tend to face a lost of inefficiencies in management and process. These inefficiencies
risk the quality of management processes. Organizations face strategic
misalignment, unreliable decision making, hidden knowledge and non-adaptability to
changing business demands.
This situation is being faced by many of the big organizations as business is purely
focused on management solutions that are based on single activity tools like
Budgeting, reporting etc.

BPM is a strategic tool, which can improve management processes by providing fact
based decision making, strategic focus, collaborative management and thereby
increase their return on investment.

References

1. Business Performance Management – Hyperion By Alistair Shaw
2. Article published in DM Direct Special Report – Feb 24,2004, By Greg
Mancuso and Al Moreno

Authors Profile

Gayatri Jayanthi is a Senior Software Engineer in the Data ware Housing and
Business Intelligence Practice. She has an over all experience of 5 years, out of
which last 4 years have been in Data ware housing exclusively on Informatica,
Business Objects and Brio. Currently she is leading a team working on Data Stage
Parallel Extender. Email Id : gayatri_jayanthi@satyam.com


Model Management
(Keshav Tripathy)

Abstract

Nowadays all IT organizations have to execute projects within challenging conditions:
Pressure on price, time, maintaining and improving the quality, managing the
resources and taking care of unforeseen risks. Hence, it becomes imperative to be
prepared for the variations of the above parameters before hand. This makes it
necessary to plan and execute the project on specific guidelines, taking the
variations of the parameters into consideration. To do so, managing the various
models associated with a project life cycle becomes crucial.

Models are representations of Functions and Information in use or planned for use by
any organization. Models may be of Business Oriented which reflect the business
activities and information or may be Information System oriented and reflect the
functions and data of a specific information system. The models are developed
keeping in mind the purpose of the project and the parameters or variables the
project is dependent on. The models are developed based on specific modeling
techniques and have a firm mathematical background to it.

Effective Model Management means ensuring that the correct model is available to
individuals within the business and the Information System groups as needed. This
call for making models available at begin of projects, incorporating changes and
providing models to the business community. It may also involve coordination of
model usage across projects and the validation that they reflect the business as it is
and that they conform to the Organization Standards.

This paper does not do is describe the various models for project management but it
discusses the model management as one of the integrated functions of project
managers. It introduces what are models, why a project needs models and what are
the various models available. The paper discusses on the importance and usage of
model management for projects by other stakeholders of the project viz. the team
and the business groups. As many people may interact with the model with various
roles as project manager, team member, customer etc, it becomes necessary to lay
out rules on the management of the model so that it’s integrity is maintained.
Overall the paper introduces to the reader the need for model management in
software development project and other issues associated with it.

1. Introduction

Creating a business solution on time and within budget requires a disciplined
approach. Most large IT organizations rely on development methodology to keep
projects on track, including some developed during the mainframe era. These older
methodologies are often too inflexible to provide value in today's distributed
Internet. But whatsoever it may be the solution space a robust Project Management
Methodology based on Modeling Technique is essential to make the Project Success.

2. What is Project Management?


Simply put, it's a way of managing complex tasks. In reality, however, project
management is both an art and a science in which a specific regimen of principles,
tools and techniques is combined with political savvy, good negotiation and people
skills.

Every company, large and small, can benefit from good project management skills.
Incorporating project management into the work environment leads to better
efficiency and increased profitability. It helps a company consistently achieve goals
and objectives. It enables a small company to compete by making better use of
limited resources. It can help a company to execute projects on time, within budget
and with consistently high quality. Project management is a proactive way to prevent
fires. In today's tough business environment, good project management based on
strong models can give a company the competitive edge.

3. What are Models

Models are representations of Functions and Information in use or planned for use by
any organization. Models may be of Business Oriented which reflect the business
activities and information or may be Information System oriented and reflect the
functions and data of a specific information system.

Normally these models are diagrams that facilitate the understanding. Such
diagrams can illustrate complex processes, such as the Flow of Information through
the business, or a simple process like as a List of activities or data groupings. The
models may be developed in many different environments and contexts: an
Information System development project, a new manufacturing system; early
models to show the major functions of the system; later in project development to
illustrate the detailed processing logic. The business may also use modeling as part
of a business analysis effort.

4. Why Does Management Need Models?

There are three primary reasons why project management needs models associated
with a systems development project:

1. To estimate the money, time, and people required to develop the project.
2. To update and revise those estimates as the project continues.
3. To manage the tasks and activities of the people working on the project.

Budget and estimates are one of the major activities of managers in any project;
they are all the more difficult in many systems development projects because each
project is (or at least seems to be) unique. However, management still needs models
— graphical models are clearer — to see the availability of the people, what will
happen if those people are not available, and so forth.
Even the best plan, though, is likely to fail if it is implemented blindly. Circumstances
change continuously during the project: critical resources not available; inaccurate


work estimates; sudden changes on the user’s dates; work completed more slowly
than originally expected. Thus, it is important for the project manager to have
models that can let her or him explore the consequences of changes in the plan.

And finally, the project manager must not only manage tasks, he or she must
manage people. The manager must ensure that all the personnel are doing what
they should be doing, when they should be doing it. The manager needs modeling
tools that will focus on the people, in addition to modeling tools that focus on the
tasks

5. Purpose of Models

5.1. Documentation: A primary purpose of models is to document the business
and the system that support the business. Such documentation, especially in
graphical form, can provide valuable information to business on how it operates.

5.2. Understanding the Business: Through the development of Business-
Oriented Models, allow a better of the business functions and then information used
to perform those functions. Understand the business problem should be the
foundation of any Information Systems work performed. and the models illustrate to
the Information Systems group how the systems support critical business functions.

5.3. Understanding an Information System: System Oriented Models help
the business and the Information Systems group better understand a specific set of
programs and databases. Visually they can identify major system functions, such as
Planning and Inventory Control within a New Manufacturing system, as well as
detailed system processing, such as the algorithm used for Production Planning.
These models also help explain

5.4. Communication Tool between the Business and Information
System: By developing common business oriented terminology communication
barriers can be reduced between a specific business unit and the information
systems group that supports it. The graphical nature of many models also assists
communication: It may require several pages of text to describe how Order
Information flows from Order Entry System to the Order processing System to Order
Fulfillment System but a simple one-page diagram may illustrate this same
information in a better way.

5.5. Improving System Quality: From the System developer’s perspective,
models can improve the quality of systems analysis and design. For example when
looking at new H.R. System from this high level it may be easy to identify that, while
hiring and terminating employees can be supported, there is no mechanism for
transferring employees. At the design level the developer can define detailed
processing logic graphically which will facilitate the identification of omissions and
inaccuracies by the developer and the other team members.


5.6. Improving Business Processes: Businesses are using models of their
business processes as the foundation for improving process efficiency, by
documenting current activities and modeling new approaches, the business can
easily identify current inefficiencies and possible solutions. This has applicability to
activity improvement, job redefinition and departmental reorganization.

5.7. Project Team Coordination: Within an Information System project team,
models can become a means of coordination. They can depict a high level overview
of how a system will function so that all team members can understand how their
area of responsibility meshes with the entire system. A more detailed model can help
the team members understand specific relationships between their activities and
those of then other team members.

5.8. Cross Team Coordination: Many of the benefits available with a Project
Team approach are also available across the Information Systems Department.
Specially in integrated system environments, it is critical that analysts understand
the way in which their system integrates with other systems. This understanding
facilitates better design and maintenance of integrative process and data.

6. Project Management Models

Project Management Models which are very successful in executing complex projects
have been identified as key to success of Modern Project Management. Some of
these are the following.

• Enterprise Architecture Model: Designed to shorten the enterprise
architecture planning cycle, this model provides guidelines for building enterprise
architecture through versioned releases.
• Team Model: Designed to improve internal teamwork, this model provides a
flexible structure for organizing teams by project. The Team Model can be scaled
to meet a project's scope, a team's size, and the team members' skills.
• Process Model: Designed to improve project control, minimize risk, improve
quality, and shorten delivery time, this model provides project structure and
guidance throughout the project life cycle. The Process model describes the
phases, milestones, activities, and deliverables of a project, and explains their
relationship to the Team Model.
• Risk Management Model: Designed to help the team identify priorities, make
well-informed and strategic decisions, and prepare for contingencies, this model
provides a structured and proactive environment of decisions and actions to
continuously assess, prioritize, and deal with project risks.
• Design Process Model: Designed to help discern the true business and user
needs behind a technology project, this model employs a user-centric continuum
to enable efficient and flexible design through a parallel and iterative approach.
The conceptual, logical, and physical design phases provide three different
perspectives for the three different audiences: users, team, and developers.
• Application Model: Designed to improve development, maintenance, and
support, this model provides a three-tiered, services-based approach to designing


and developing. The services utilized in this model are scaleable, and can be used
on a single desktop or on multiple servers worldwide.

These models promote principles such as risk-driven scheduling, a fixed-release-date
mind-set, versioned releases, visible milestones, and small, peer-based teams.
Modern Project Management exposes critical risks, important planning assumptions,
and key interdependencies required for successfully planning, building, deploying,
and managing the company's mission-critical technology infrastructure or business
solution.

7. Why Model Management

In any organization small isolated groups start developing project specific models.
Gradually more groups begin to develop project specific models and then people
start to edit some other person’s model. At this point the organization needs some
form of Model Management.
Effective Model Management means ensuring that the correct model is available
to individuals within the business and the Information System groups as needed.
This call for making models available at begin of projects, incorporating changes
and providing models to the business community. It may also involve
coordination of model usage across projects and the validation that they reflect
the business as it is and that they conform to the Organization Standards.

8. Challenges of Model Management

If you see from the surface it seems like a relatively simple task as because it is the
one that information system personnel have been dealing with success for several
years in the area of program source management (i.e. Source Control or Version
Control), Data Dictionary Administration etc. However a Model Manager must deal
with some unique problems.

• They must manage many different types of objects with complex relationships.
The model management deals with Program Sources, Data Dictionary and also
manages all the process-related objects, possible planning objects and the
relationship between the objects.
• Unlike Data Objects and Dictionary and Programs, most model objects have
limited standardization. Different vendors have come up with separate models in
a different way. Each methodology may dictate the use of a different set of
modeling objects or different object relationship. Same models can be depicted in
different ways. Therefore determining what constitute a real change to a model is
difficult.

9. Types of Model Management

• Project Model Management: The Project Model Management involves in
tracking the development of models for a specific projects. This type of Model


Management is restricted in scope. The challenge here is to ensure that valid
models are available to all Team Members.
• Corporate Model Management: This is to manage the models across all
Projects in the Organization. It also involves management of Models that are not
currently a part of the project but are of interest to the Organization.

The given Figure 1 below describes Project and Corporate Model Management as
separate activities.

Fig: 1

Fig: 1

10. Functions of Model Management

10.1 Establishing and Enforcing Standards

One of the primary functions of model management is the establishment and
enforcement of model standards. To develop these standards the model manager
must first determine what types of objects are required and what types are optional
for the models. Required and optional relationships between objects should also be
identified. For each of these objects the model manager must identify the
information that should be collected. A critical part of this detailed information is the
name of the object. For objects that represent the physical pieces of the system, the
physical naming standards can be used. New naming standards must be developed
for other objects.

10.2 Establishing the Organization’s Information Model

An approach to documenting object-related standards is through an information
model. It is a diagram that shows the required and optional objects and relationships
for the organization. A more detailed model may also include specific properties such
as name and definition that are also tracked about an object.

Figure 2 below shows a sample information Model. This is a high-level information
model.


Fig: 2

10.3 Establishing and Enforcing Procedures

Once the standards are determined, model managers need to identify how the
standards will be enforced. Procedures identify how and when models will be
updated. It is also helpful to identify clearly who is responsible for performing the
updates. If corporate models are being developed it is also important to determine
how project models will be merged together to form corporate models.

In establishing model maintenance procedures the model manager needs to identify
all possible points where models can be changed. Information System Development
Projects are one obvious source of change. Business changes, system maintenance
and planning projects are also possible source of change. In each of these scenarios
the model manager needs to identify how the model will be changed and who will
perform the update.

10.4 Establishing the Organization’s Activity Model

Model management procedures can also be documented through a model. While the
information model documents the types of information to be managed, the activity
model documents the steps for managing those models. An activity model may be at
a very high level identifying the major functions to be performed or at a very
detailed level documenting the procedures themselves.

Figure 3 & 4 below depict a sample activity Model. First one depicts a Process Model
decomposition diagram format. The second one illustrates which portion of the
information model is affected by each activity.


Fig: 3

Fig: 4

10.5 Tracking Model Updates

Both the standards and procedures should include a mechanism that helps track
model maintenance. Tracking involves two activities: identifying that a change has
occurred and reacting appropriately to the change. Model managers must ensure
that all updates that result from the changes are completed. Once a change has been
identified, the model manager should also ensure that all associated changes have
occurred.

EXAMPLE: If a business process has been changed, the programs that support that
business process should also be changed accordingly. Although the model manager


is not responsible for making such changes but he should inform the individuals who
maintain these programs. This requires tracking of not only the objects themselves
but also the linkages between objects and the individuals who have the responsibility
of maintenance.

11. Project Level Model Management

The most basic form of model management occurs within a project team. On a team
with one individual, the developer is responsible for ensuring that all required objects
are created and are linked appropriately to provide them information necessary for
efficient and effective application development. However, when more than one
individual is involved in the model development activities for the project, model
management becomes more complex task. The work of all developers must be
coordinated to ensure that the resulting models accurately combine to represent
both the business and the system that support it.

Critical issues in project level model management include assigning responsibility for
objects and linkages to the team members, coordinating the use of shared objects,
and resolving conflicts within the model. It is also important to identify clearly for the
team which objects and linkages are required. The team can then identify which
diagrams and modeling techniques will best help them meet model management and
project communication requirements.

12. Conclusion

We have seen that how crucial is the function of a Model Manager in a Project and
Model Management activity must be aligned with the Project Management activity.
Every Project Managers must take into this consideration for managing large projects
which are more of developmental in nature. By ensuring proper model management
in a Project one can ensure integrity of the models across all the phases of the
project as well as it will provide seamless integration with different modules of the
application systems.

13. References

1. Software Project Management: A Unified Framework: Walker Royce, Addison
Wesley
2. Software Engineering Economics: Boehm, Barry W: Prentice Hall Englewood
Cliffs, NJ 1981
3. Industrial Strength Management Strategies: Brown Norm, IEEE Software V-13
N-4 July 1996
4. Edward Yourdon’s Modeling Tools for Project Management.
5. Database Modeling and Design: Tody J Teorey, Morgan Kaufman Publication

Author Profile

Keshav Tripathy – Has been with Satyam from February 2001. Currently he is
working with EABIS (BI &DW)Circle, Bhubaneswar.. Keshav_Tripathy@satyam.com


Data Warehouse Tuning
Whenever a system matures and performance starts to become an issue, there is an
activity known as “Tuning the system” that assumes importance. When a system is
tuned, it yields better performance without any major and wholesale changes to the
system. Tuning the system has been a time honored system management activity
ever since the first information system was built.

What is Data Warehouse Tuning?
Tuning the Data Warehouse / DSS environment – for the purpose of this article – is
taken to mean the activities that will enhance performance that stop short of a
wholesale rewrite and reconstruction of the data warehouse. It also means the
removal of the data warehouse off one hardware platform and onto another
platform.
Tuning begins with the monitoring of the activity and the data that flows through the
data warehouse and the DSS environment. If there is no monitoring, then finding the
exact cause of the slow performance will be a difficult task. In order to tune the
system the Data Warehouse administrator must know what is going on in the
system. Otherwise tuning is merely a guessing exercise.
A monitor indicates the administrator what is going on inside the system. It also tells
the DWA how effective the attempts at tuning have been, once the changes are
made inside the data warehouse. Therefore, it is essential that both activity and data
be monitored before the tuning exercise commences and during the tuning exercise
itself.

Operational Database Tuning Vs. Data Warehouse Tuning
Tuning the system takes on a different meaning and importance in the Data
Warehouse / DSS environment than in the classical operational environment. The
first reason why tuning is different is that the essence of the data warehouse
environment is one of iterative development.
Unlike the operational environment where system requirements are known prior to
the building of the system, in a data warehouse environment the developer often
does not know what the requirements are going to be until the system is built. Only
after the first iteration of the data warehouse is up and running does the developer
know for sure what is going on inside the system. This uncertainty ensures tuning
the data ware house / DSS environment system after the initial construction is
absolutely mandatory.
A second tuning difference between the operational environment and the data
warehouse / DSS environment is that of the fundamental nature of a transaction.
Operational transactions are mission critical and require 2 - 3 seconds response time.
Operational transactions require a higher degree of availability and each transaction
uses only a small amount of system resources. Data Warehouse/ DSS transactions
are not mission critical and often use a vast amount of resources. It is normal for


many types of DSS transactions to have a longer response time. Furthermore, given
the fact that DSS transactions are used for long term planning, not for immediate
business decisions, the business case for a very fast response time is weak.
Therefore, the acute degree of software engineering required of operational
transactions is not required for data warehouse / DSS transactions. In this sense,
tuning the data warehouse / DSS environment is not quite critical as tuning the
operational environment.
Another factor relevant to tuning is the volume of data found in each of the
environment. The data warehouse / DSS environment contains 1 - 2 more orders of
magnitude of data than the operational environment. This difference in the volume of
data requires the DWA to possess sophisticated management skills, of which tuning
is an important part.

Tuning Activities
Creation of Indexes
The simplest tuning activity is that of adding indexes to data based on access and
analysis.. When used properly, indexes can reduce the system resources required to
locate data. However, a trade off is to be made. Once the index is specified, the
maintenance that is required every time data is added to the data warehouse must
be accounted for. In addition, indexes require their own space. But for data that is
accessed frequently down the same path, indexes can save processing resources far
beyond their cost.
Creating Data Marts
Creating data marts is a powerful way to enhance performance. With a data mart,
the end user specifies and extracts a selective subset of data from the data
warehouse that is customized to meet particular departmental needs. With a data
mart, the volume of data can be tremendously reduced. Data can be summarized,
and customized. All of these factors lead to a much greater performance in the eyes
of the end user.
In addition, data is isolated on a machine dedicated to the individual needs of a
department. With this isolation, the department can achieve a boost in performance
as they are willing to pay for. The cost of the machines used at the data mart level is
significantly lower than the cost of the machines used at the data warehouse level.
For these reasons, data marts are a popular approach to achieving performance.
Summary Tables
An alternative approach to data marts is to create organization wide summary
tables. These summary tables are open and available to any end user on the data
warehouse and not just to the user of the data mart. The summary data is much
faster to access than the base data that must be recalculated every time the
summary is desired. Therefore, the summary data requires space. This is the case
with every unit of summary data, there is a process of summarization. This process
of summarization becomes “meta process” information that is necessary to describe
the basic nature of the summary data itself.


Removing Dormant Data
Dormant data is data that is not accessed but resides in the data warehouse.
Removing dormant data greatly enhances the prospects for performance. Dormant
data takes up space and gets in the way of serious DSS analysis. If data is not being
used, then it has no place in the data warehouse / DSS environment. The effect of
dormant data (or the presence of dormant data) is normally not felt until the
warehouse is fairly large and mature. In the early days of a data warehouse, almost
all data is being used. Even if the data is not being used, the small dimensions of the
early data are such that dormant data hides with no ill effects. But as a data
warehouse grows larger, the percentage of the data that is not being used increases
drastically. When a data warehouse contains more than 50% of dormant data, then
it is time to prune the data warehouse of the data that is not being used.
Star Joins / Snowflake Structures
Another very useful technique to enhance performance is to create STAR JOINS.
STAR JOINS for their larger counterpart – SNOWFLAKE structures are useful for
streamlining processing. In a STAR JOIN there is a FACT table and several
DIMENSION tables. The FACT table is pre joined to the DIMENSION tables to
accommodate access to the data. While FACT tables can enhance performance, they
must be used with caution. The first problem with STAR JOINS is that they require a
foreknowledge of how the data is to be used. If there is foreknowledge, then FACT
tables and DIMENSION tables make a great deal of sense. But if there is no
foreknowledge of how the data will be used, there is no guarantee that the STAR
JOIN will accomplish a boost in performance. A second issue is that one group of
people may have a single pattern of usage while another group of users may have
another pattern of usage. The STAR JOIN can optimize access based on only one
pattern of access. For this reason, STAR JOINS are best equipped to optimize the
performance of large data marts, where there is a better chance of a homogeneous
usage of data.

Data Partitioning
By partitioning the data finely at the database, there is the chance that a higher
degree of parallelization can be reached in processing data. For example partitioning
tables based on time id for each and every FACT tables enhances the performance of
the query by reducing the time taken to locate the data to a great extent.
Defragmentation of The Table spaces
In a typical Data warehouse / DSS environment running on Oracle Database, space
management is very crucial in maximizing performance as well as space usage of a
database. Over a time period, the database can become fragmented affecting the
performance and resources of the database. If any ORA- errors regarding allocation
of extents or space is received and the DBA_FREE_SPACE reveals a lot of free space
left in that tablespace, then the database on which the warehouse is running is a
victim of fragmentation.
When a tablespace is fragmented, there may be a lot of free space in the tablespace,
but it is in such small pieces that Oracle cannot use them. When an object needs a
next extent, Oracle has to allocate one contiguous extent. If one does not have a
chunk of free space that is large enough for the extent, Oracle will return an error. A
tablespace may have smaller pieces of free space that may add up to the size of the


next extent or even more, but Oracle cannot divide the next extent of the object into
smaller pieces to fit into the free space. De allocation of extents from an object to
the free list will cause a tablespace to become fragmented. The more often this
movement of space occurs, the more fragmented the tablespace will become. This
phenomenon can be more prominent in a data-warehousing environment, where
many tables and indexes are being dropped, and reloaded.
There are two types of fragmentation that may be seen in a tablespace.
• A tablespace may have 2 pieces of free space but in between the two, there is an
extent belonging to another permanent object. There is no way to combine these
two free extents and repair the fragmentation.
• Another type of fragmentation occurs when the tablespace has two pieces of free
space that are adjacent to each other, but they are separate. This type of
fragmentation can be repaired, by manually combining the tablespaces.

The only way to get rid of fragmentation within an object is to drop and recreate the
object. However one can take care of the second problem mentioned above by
manually combining the tablespaces.

Separating FARMERS from EXPLORERS
One of the useful classifications of users in the data warehouse / DSS environment is
that of explorers and farmers. Explorers are those people who do not know what
they want but will know it when they find it. Farmers are those people who regularly
access data having a good idea of what they want. Both types of users are valid and
are normal in the data warehouse / DSS environment.
Separating FARMERS from EXPLORERS is one of the easiest and most effective
ways to enhance performance.
Farmers submit small requests that access a uniform and limited amount of data.
Furthermore they also have a regular pattern of access. They almost always find
what they are looking for. Explorers do not know what they are looking for and
submit requests on a very irregular basis. Explorers look at huge amounts of data
and often find nothing.
The pattern of data and system usage created by farmers and explorers is very
different. The patterns are eternally in conflict with each other in terms of system
utilization.
The Data warehouse administrator can enhance performance for everyone by
separating the transactions submitted by each community. The explorer transactions
are run after hours and the transactions the farmers submit are run during the day.
In doing so, the workload starts to achieve some degree of predictability.

Tuning the Data Warehouse according to its usage
In order to fine tune the system the Data Warehouse administrator must keep track
of what exactly is happening in the system. He must be able to answer the following
questions based on the day to day tracking of the Data Warehouse


• What data is being used? How often, by whom and When?
• What is not being used? Is it ever being used?
• Who is using the Data Warehouse? Should they be there?
• Who is not using the system and who should be?
• What is the most resource – intensive operations on the system?
• What is the average response time of queries across the system?

Asking these questions is the first step towards improving the data warehouse. It is
the aftermath of a successful data warehouse implementation that there are
questions generally asked by people contemplating the next generation data
warehouse.
There are different approaches to getting these usage questions answered. Each
approach has its own assumptions, tools and methods. The basic intent behind the
strategy to monitor, record, analyze and correct is also known as usage tracking.
There are three general mechanisms for usage tracking.
• Purchase or build end user tools which record usage and present it for analysis
• Sample usage information from the database’s internal views
• Intercept, record and analyze (all or selected) network traffic.

Once these questions are answered to some extent the cause of the slow
performance can be located and thus subsequent corrective measures can be taken.
Eliminating “HOT SPOTS”
Of course, where data resides upon parallel hardware architecture, identifying and
removing “hot spots” can produce great gains in performance. Even when hot spots
can be located, there is no guarantee that they will reappear. Yet, being able to
identify hot spots in a valuable piece of information to the DWA, charged with
producing optimal performance.

SUMMARY
Tuning the Data Warehouse is fundamentally different from tuning an operational
environment. There are many techniques to the elimination of data that is not
required or the unnecessary processing of data for the purpose of DSS processing.
Some of those techniques include:
• Creation of indexes
• Creation of Data Marts for different subject areas
• Creating SUMMARY tables
• Removing Dormant Data
• Star Joins / Snowflake structures
• Data Partitioning
• Defragmentation of the tablespaces
• Separating Farmers from Explorers
• Tuning the Data Warehouse according to its Usage
• Eliminating “Hot Spots”

Article Contributed by Knowledge Dynamics a unit of Satyam BIDW.


Dw Quest+Oct+2005

Recommended

Recommended

More Related Content

Similar to Dw Quest+Oct+2005

Similar to Dw Quest+Oct+2005 (20)

More from keshav2011

More from keshav2011 (13)

Dw Quest+Oct+2005