Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Optimizing Organizational Performance by Managing Project Benefits


Published on

Too many organizations today still measure the success of a project based only on the traditional project management standards of delivering On Time, On Budget and On Scope. While these criteria are …

Too many organizations today still measure the success of a project based only on the traditional project management standards of delivering On Time, On Budget and On Scope. While these criteria are valid measures of successful project management, they are less suitable when assessing a project’s true success: its contribution to the overall organization's performance. Indeed, the ulti-mate success of a project – whether cost savings, revenue increases or customer satisfaction improvements – may not be known until years after it has been successfully delivered.

Published in: Business, Technology

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Optimizing Organizational Performance by Managing Project Benefits A UMT White Paper By Yorai Linenberg, Ziv Stadler, and Stephen Arbuthnot
  • 2. <2> Introduction Too many organizations today still measure the success of a project based only on the traditional project management standards of delivering On Time, On Budget and On Scope. While these criteria are valid measures of successful project management, they are less suitable when assessing a project’s true success: its contribution to the overall organization's performance. Indeed, the ulti- mate success of a project – whether cost savings, revenue increases or customer satisfaction improvements – may not be known until years after it has been successfully delivered. The challenge senior IT and business managers face today, however, is not only to successfully manage the project throughout its lifecycle, but to quantify upfront how a project’s success, or the benefits it generates, can be measured. Once these benefits have been committed to, they need to be tracked throughout the post implementation phase in order to ensure that the project pro- vides the greatest strategic value to the organization. By identifying and remaining cognizant of a project’s benefits, organizations can make great strides towards ensuring they are maximizing its ultimate success. Optimizing an organization's performance, however, needs to begin long before individual projects’ benefits are defined, their exe- cution managed, and their delivery tracked. Rather, the organization's decision-makers must first ensure that they have chosen the most appropriate projects for inclusion in the portfolio. Even the most successful project delivery will not help to optimize the or- ganization's overall performance if the projects chosen for the portfolio are not aligned to the business strategy. A project benefits management framework should therefore be viewed as an essential step towards improving the efficiency of a portfolio of projects, and ensuring that the expected benefits are aligned with the business strategy, and in fact, being realized. Such a framework natu- rally complements, and should be integrated within, an overall project portfolio management (PPM) approach, which views a portfolio of pro- jects in the same way as a portfolio of investments. The ultimate, long term objective of both is obviously to increase the portfolio owner’s (whether an individual or organization) return on investment (ROI). There are three main differences, however, between these portfolio types, which explain why such a framework is so important: 1. In the short term, it is not always possible to measure a project’s financial contribution; therefore additional measurable benefits must be identified. 2. Whilst the performance of a financial portfolio can be measured at any point in time, the success of a project, and its contribution to the organization's strategy, can often only be measured long after it has been delivered. 3. While the contribution of each security to the overall performance of a financial portfolio can be readily identified, due to inter- project dependencies and overlaps in scope, a measurable organi- zation benefit is usually the outcome of several projects. By applying the same units of measurement to both the organization's measurable objectives and the individual projects’ benefits, the steps involved in implementing a benefits management framework create a visible link between the two. At a high level these steps include: (1) Formulating and prioritizing the organization's objectives, and defining their relevant key performance indicators (KPIs) and targets; (2) Identifying each individual project’s benefits and determining which ones are best positioned to contribute to the ob- jectives’ KPI target; (3) Rationalizing and optimizing the project portfolio to ensure the aggregated project benefits achieve the objectives’ target KPI; (4) Sequencing the optimized project portfolio to generate the benefits realization timeline, taking into ac- count resource constraints, dependencies and critical milestones; (5) Managing scope and timeline changes throughout the pro- ject’s execution and evaluating their impact on the benefits and their delivery timeline; and (6) Monitoring the project benefits Business’ Desired Objectives AbilitytochoosetheRightProjects 100% 80% - Choosing the best projects X 75% - Achieving the desired benefits 60% - Business’ Realized Objectives 100% Exhibit 1 – Doing the Right Projects Right Recent findings indicate that organizations did not achieve their targets because many projects were not aligned with the business objectives, and of those that were, a high portion did not realize their intended benefits. The Standish Group, 2001
  • 3. <3> realization phase, which usually stretches beyond the project’s “official” end date, to verify project success and ensure the organi- zation's objectives are being achieved. Step 1: Formulating the Organization's Objectives An organization's first step towards optimizing the performance of its portfolio of projects is to identify and define its key objec- tives. Objectives must to be specific in scope, action-oriented, able to serve as high level goals for an individual project, and should ideally have been agreed upon through a consensus approach by as wide a group of senior managers and stakeholders as realisti- cally possible. Objectives are typically categorized into three areas: demand management (e.g. sales effectiveness, increased reve- nue), supply management (e.g. operational efficiency, customer responsiveness) or support services (e.g. infrastructure, regulatory requirements). Most importantly, however, objectives need to be measurable: each one should have a set of one or more KPIs which, along with their defined targets, will enable decision makers to monitor and control the successful delivery of the objectives in real terms. KPIs: Measuring Objectives in Real Terms Defining the “right” KPIs is one of the most difficult tasks an organization faces when establishing a benefits management frame- work. Effective KPIs must be relevant to their associated objective, specific, readily measurable by the organization, realistic and achievable, and contain a time element. That a KPI must be relevant to the objective it is striving to measure may seem self- evident, but this point still needs to be highlighted. It can often be convenient for an organization to employ an existing perfor- mance metric to measure the impact on a recently defined objective. If the price of this convenience, however, is the undermining of the accurate measurement of the achievement of the objective, then a new KPI should be chosen. The KPI also needs to be specific enough so that it can measure the delivery of an individual objective, without being excessively affected by external noise and unrelated events. For example, how can a KPI as broad as a customer satisfaction index measure the direct impact of a specific project? Clearly this is when the science becomes less exacting, but it highlights the effort needed to en- sure the KPIs are well defined. That a KPI must be measurable is often overlooked in the early stages of such an exercise. It may seem obvious that the best meas- ure of increased customer satisfaction should be a higher score on a customer satisfaction survey, however if no such metric is cur- rently being tracked, that KPI may not be the most appropriate. A better KPI might be to measure the number of calls to a help line over a specific period. A side effect of a PPM exercise, however, should not be a host of new projects whose sole purpose is to de- velop processes to measure the success of other projects! As with any goal, the target KPI must be realistic and achievable for it to be relevant. There is no point in setting up targets which are beyond the organization's, or, for that matter, the market’s ability to fulfill. The final requisite characteristic of a KPI is that it contains a time element. In simple terms, the KPI must measure not only how well the organization is doing in terms of achieving its objectives, but it must also provide the time dimension for achieving these objectives. Prioritizing the Organization's Objectives Once the objectives have been agreed and the KPI’s defined, and accepting that for a given period (e.g., 12 or 18 months) not all objectives may carry the same importance, a prioritization exercise must next occur. For example, after analyzing market condi- tions, an organization may wish to focus its efforts (and initiate projects) to reduce its cost-base, rather than to increase its reve- nues. “Portfolio management takes a holistic view of a company's overall IT strategy. Both IT and business leaders vet project proposals by matching them with the company's strategic objectives.”, Portfolio Management - How to do it Right
  • 4. <4> The prioritization process is therefore used to determine the relative importance of the objectives and to generate a normalized weights vector. UMT uses the Pairwise Comparison method, which ranks the objectives in pairs against one another, in order to generate these values. Such an exercise also has the additional advantage of creating transparency and achieving consensus among decision makers, by forcing them to choose and explain their priorities. In some instances, however, the resulting discussion might be heavily depend- ent on the organizational culture and level of openness. Step 2: Identifying Project Benefits and Prioritizing the Portfolio The KPIs described in the previous section are necessarily closely linked to project benefits. Whereas a KPI measures progress to- wards achieving an objective, a benefit is the measurable, positive outcome of a project that a KPI is measuring. Certainly projects will also provide benefits that are not measured by one of the KPIs, but when analyzing projects from a portfolio view, in the con- text of an organization's objectives, we should only be concerned with the benefits that can be measured by these KPIs. The challenge is therefore to create a mechanism that will link, using the same units of measurement, the individual project’s bene- fits to the objectives’ KPIs. Such a framework will not only enable the organization to pinpoint potential gaps in its portfolio (where the consolidated projects’ benefits do not add up to the objectives’ target KPI), but will also provide a way of prioritizing the entire project portfolio against its objectives. Linking Projects to Objectives Similar to agreeing on the relative weighting of objectives, a consensus approach should be used to ensure that all the key stake- holders agree on the contribution each project makes towards achieving each objective. Therefore, once each proposed project has been defined – including an estimate of cost (both by resource-type and monetary measure), scope, schedule, dependency, and other potential characteristics specific to the organization – it must then be appraised based on the contribution, if any, that its benefits make towards the achievement of each objective. Examples for project benefits are “reducing headcount in the ac- counting department by two (2) FTEs”, or “reduce error rate in transaction processing by 10%”. In creating such a mechanism, it is necessary to establish KPI bands, or threshold levels, that will enable project sponsors to deter- mine how strongly their project supports each one of the objectives and their KPIs. UMT uses a structured approach, usually as part of a redesigned portfolio planning or project approval process, for defining these bands and assessing the contribution of each pro- ject. The KPI threshold levels are translated into numerical bands (usually 5), where “Extreme” might mean that a project will con- tribute 10%-12% of the target KPI, “Strong” 8%-10%, etc. The definition of these bands usually depends on the number of projects that can show a link to a given KPI. Evaluate the Relative Importance of Each Business Objective Exhibit 2 - Prioritizing the Organization's Objectives Review and Agree on the Business Objectives and Priorities
  • 5. <4> Prioritizing Projects: The Project’s ‘Strategic Value’ Once each project has been evaluated against the objectives and their KPIs, it can be assigned a “Strategic Value”. This value repre- sents the level of support a project is expected to provide to the organization's objectives, and is calculated using (a) the weights of the relevant objectives the project is supporting and (b) the “strength” of this support (i.e., Extreme, Strong, etc.). The strategic value may then be used as a proxy for the priority of the project within the organization's portfolio. It is important to note that, when using this approach within a project portfolio management framework, such a methodical pro- cess generally identifies project benefits that contribute to the achievement of as many of the organization's objectives as possible. However, when organizations endeavor to conduct ad hoc project benefits management exercises independently from a PPM framework, they often focus exclusively on financial benefits at the expense of ignoring others that also contribute to the overall performance of the organization. It is generally accepted that over the long term, the primary justification for a project is its finan- cial impact on the bottom line; however, since in the short term this impact is not always visible, other measurable project benefit types must be considered. These may include non-financial (e.g. quality of service, workforce motivation and satisfaction, and man- datory or legal requirements), or indirect ones (e.g. strategic fit, risk reduction, and internal management efficiency). Analyzing project benefits within the context of a portfolio, therefore, serves as a powerful tool for organizations determined to optimize their performance. Step 3: Rationalizing and Optimizing the Project Portfolio Having agreed on the organization's definition of performance, and having assessed how each project in the portfolio individually contributes to this performance, the next challenge is to rationalize and optimize the project portfolio. Rationalizing the portfolio is necessary in order to ensure it does not contain multiple projects that are conspicuously superfluous, or others that are clearly mutually exclusive. Potential overlaps between projects should be exposed; for example, projects from different business units may appear to achieve the same benefit, in the same functional area, by using different approaches. These overlaps and duplications should be highlighted as alternatives or removed. This step obviously requires detailed knowledge and understanding of the projects and their benefits. Assess Each Project’s Contribution to the Overall Target KPI Prioritized Project List Based on “Strategic Value” Exhibit 3 – Linking Projects to Objectives “Rather than trying to calculate specific returns on each project, some companies are saving millions of dollars each year through a portfolio approach to IT spending.” Computerworld, Stabilizing Your Risk
  • 6. <4> Once potential duplications have been dealt with, each category of benefits should be consolidated to evaluate whether the target KPIs can be achieved. If gaps are identified, then decision must be made to either amend the scope of existing proposed projects or propose new projects, in an effort to create a portfolio of projects that has the potential to achieve each objectives target level of KPI. Although the organization will strive to achieve all its target KPIs, in reality, this is not always achievable from an operational per- spective. A portfolio optimization step will therefore be required, which will take into consideration the usual constraints faced by IT managers: money, resources and their types, time, level of risk, etc. In a perfect world, with no constraints, the entire rational- ized portfolio would be implemented, but in the real world, and especially in these economically challenging times, difficult deci- sions need to be made in order to create the optimal project portfolio that delivers the maximum strategic value. UMT uses various algorithms to create an optimal portfolio of projects that yields the greatest strategic value, no matter what con- straints are applied. It is important to remember, however, that this optimized portfolio does not necessarily “select” the projects with the individual highest strategic value, but rather it proposes the overall portfolio with the highest aggregate strategic value. It also does not necessarily select projects with the greatest ROI, or net contribution to increased revenues – traditional characteris- tics of approved projects – unless of course a heavily weighted objective’s KPI is increased revenue or ROI. Once a project has earned its place in the optimized portfolio, following the prioritization and optimization steps, this phase’s final step is for each project’s benefit realization schedule to be formally signed-off and committed. The importance of this step can not be overstated, as it sets forth the commitments made by the project owners and managers regarding the project’s delivery. This is also when the project’s benefits realization schedule – the manifestation of what the KPIs will be measuring – is proposed, approved and therefore committed. The benefits realization schedule is what will be tracked dur- ing a project’s execution and post-implementation phases to measure whether the benefits proposed during the approval phase are indeed being achieved. In the context of most organizations' traditional project lifecycle, this would often happen during the project approval process, when a formal assessment is made to determine whether a project should be funded. While the level of documentation will vary depending on the culture of the organization and the type of project, the underlying deliverable usually takes the form of a busi- ness case, outlining both project costs and benefits. Step 4: Sequencing the Optimal Portfolio Once the optimal project portfolio has been selected, the next challenge is to create a roadmap for delivering the selected projects, which, in turn, will define the overall benefit realization roadmap. Ideally, all selected projects could be fully staffed and kicked off immediately. In reality, however, an organization's supply of resource types rarely matches the demand. Furthermore, additional complications result from inter-project dependencies, external dependencies, critical milestones, etc. A project sequencing analysis needs to be undertaken, therefore, to understand the resource gaps and the tradeoffs an organiza- tion faces between different alternatives. For example, the costs of staffing up all projects based on the original timeline – where- by expensive contractors or permanent staff may be hired during some busy project phases, which are then followed by extended periods of resource underutilization - should be measured against the alternative of smoothing internal resource utilization volatili- ty. This latter scenario may result, however, a delay to some projects, which will cause their specific benefits to be delayed; the consequential knock-on effect will result in a delay in the achievement of the organization's overall target KPIs. UMT uses dedicated software tools to enable IT managers to run “what-if” scenarios either to optimize the scheduling of projects while keeping resource-type surpluses and shortages as flat as possible, or alternatively to keep a rigid schedule but identify clearly expected resource gaps. “More sophisticated analytical tools are offered by vendors such as United Management Technologies… providing actionable measurements of variables such as cost, risk and resources to help IT organizations maximize their IT Investments.” Computerworld, Balancing the IT Portfolio
  • 7. <4> Step 5: Managing Changes throughout the Project’s Execution While a project is being executed, there are a number of activities that must take place to ensure the portfolio’s project benefits will indeed be realized. Firstly, each project’s approved benefits realization schedule – which was committed when the business case was approved – must be validated at regular intervals, usually during the course of project status reporting. Regular, timely, and effective project status reporting is a vital tool to provide early warning of an under-performing project, the effect of which may likely impact the benefits realization schedule. More importantly, however, the impact of changes to the project must be fully taken into consideration. While conventional program management change control is a trademark of any well-managed program, it is commonplace for scant attention to be paid to assessing how changes may impact project benefits. This is a significant oversight, however, since the true success of a project cannot be measured without ensuring its proposed and signed-off benefits have been realized. Since a project’s signed-off, expected benefits will be directly impacted by any changes to scope or timing, any changes to these two parameters during execution need to be assessed through a formal change control process to identify the knock-on effect to the overall benefits realization schedule. Furthermore, if significant changes to a project do occur during the execution phase, and these have measurable impacts on the benefits realization schedule – both magnitude of benefit achieved and timing of realization – corrective action should be consid- ered by the project owner and stakeholders. If necessary, a decision to cancel the project may need to be made if it is determined that organizational resources could be better deployed in such a way to maximize strategic value. Finally, while changes to a project’s scope and timeline are generally within the organization's control, external events certainly are not. If significant external events occur, they may have an impact on the organization's objectives, priorities and target KPIs and a similar assessment of the project(s), within the context of the portfolio, should be conducted in order to determine what corrective action may need to be made to ensure the expected benefits and their delivery timeframe are maintained. The assessment should also take into account new projects that may need to be integrated into the current portfolio. Step 6: Monitoring the Project Benefits Realization Phase For most organizations, once a project has been implemented and the project team has been dismantled and redeployed across new initiatives, the project is considered effectively closed. Many organizations wisely conduct a post-implementation review to ensure all mistakes and shortcomings experienced during the project are documented and presumably avoided in the future, but effectively no more attention is paid to the project. This approach is a mistake. The success of a project, and hence its strategic value and contribution to overall organizational perfor- mance, can only be truly declared once it has reached its Goal Achieved Status: the state when all the benefits identified in the business case have been achieved. Only at this point, when the relevant KPIs indicate that the expected impact on the objectives has been met, can a project be considered successfully completed. Indeed, for many organizations it may be hard to identify the specific contribution a project has on the organization's performance. Ideally, in order to track the project benefits during the post-implementation benefits realization phase, an organization should have in place a process to clearly define the benefits and a process to collect the benefits data and report on them regularly both by individual project, as well as for the overall portfolio, which in effect provides a scorecard of the organization's performance. This final step in the benefits management framework provides a powerful report card view of the organization's actual perfor- mance today, against the one that was proposed in the past when the original decisions were made. As such, difficult and often contentious project investment decisions made in the past can be re-visited in the future and definitively evaluated for their judi- ciousness. Good decisions are validated, while bad decisions are identified so that corrective action may be taken to ensure they are not repeated.
  • 8. Conclusion The bad news is that any organization committed to optimizing its performance faces a myriad of options and will always be forced to make difficult decisions. The good news, however, is that a holistic approach and a structured decision-making frame- work help eliminate the bad options and illuminate the right decisions. A framework for project portfolio management in general, and benefits management more specifically, brightens senior man- agement’s decision-making spotlight and focuses it on the portfolio that delivers the mix of project benefits that brings the great- est strategic value to the organization. Key points to achieving success include:  A delivered project – even though it may be on time, on scope, and even under-budget - does not equal a successful pro- ject.  Doing the right projects is as important doing the projects right.  Strategic value, not ROI or NPV, is the currency of organizational performance.  The only relevant project benefits are the ones that contribute to achieving organizational objectives.  Decisions to select, change or kill a project must be made at the portfolio level; this is the only view that can see the im- pact on the organization's performance.  Rationalizing a portfolio can help save a lot of money by identifying duplications and overlaps; in parallel, this process also highlights gaps which may result the organization's objectives not being achieved.  A project’s benefits realization schedule is fickle: it needs to continually be revisited and validated because any little change to the project will upset it.  The Go Live date signals the start of the benefits realization phase, not the end of the project.  Even a failed project can have some benefits; the same mistakes shouldn’t be made again. References: 1. The Standish Group. (2001) CHAOS Report. West Yarmouth, MA 2. (May 2003), Portfolio Management – How to do it Right, Todd Datz 3. Computerworld (May 2001); Stabilizing Your Risk, Robert L. Scheier 4. Computerworld (Feb. 2003 ) Balancing the IT Portfolio, Thomas Hoffman # # # 1 Battery Park Plaza, 4th Floor, New York, NY 10004, (212) 965-0550,