Project Portfolio Delivery Effectiveness Feedback Example Report
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Project Portfolio Delivery Effectiveness Feedback Example Report



This is an example report you will receive about the status of your project portfolio free of charge if you complete our 15 minute survey. TRY IT!

This is an example report you will receive about the status of your project portfolio free of charge if you complete our 15 minute survey. TRY IT!



Total Views
Views on SlideShare
Embed Views



4 Embeds 24 18 4 1 1


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Project Portfolio Delivery Effectiveness Feedback Example Report Presentation Transcript

  • 1. Project portfolio delivery effectiveness survey ACME feedback report Tim Pare, John Hall, John Kirkwood January 17, 2009
  • 2. Foreword
    • In organizations that deliver a portfolio of many projects, there is perpetual pressure to do it better. This might be because their customers want to focus effort on more of the ‘right things’, do more with the same (or less) or improve speed and reliability of delivery.
    • It is clear that there are many drivers that can contribute to, or impede, overall delivery performance, spanning the fields of governance, portfolio, project, financial, workforce, resource and financial management.
    • Many emerging maturity models are predicated on the assumption that ‘best practice’ is an absolute, and if sufficient of the many prescribed practices are documented and complied with, an organization’s outcomes will be better. While this might be true, it is rather undiscriminating and might imply a huge and expensive challenge for an organization to contemplate.
    • We have observed that there usually isn't much guidance on the prioritization of prescribed practices and usually no direct matching on how those practices can help deliver an organization's specific improvement goals. Equally, the focus of the practices is generally on the ‘harder’ elements (organization, process and standards), with insufficient emphasis on the ‘softer’, capability aspects (the really difficult stuff – insightful intelligence, strong decision making, political influence and personal credibility).
    • We contend that a more succinct definition of best practice in this arena (as others) would be a set of practices that are ‘fit for purpose’. There are a lot of things an organisation could do; the real question is what they should do.
    • This report provides the results of ACME’s responses to our diagnostic and contrasts them with findings of other respondent organizations (anonymized for reasons of confidentiality) and wider research into those characteristics and key elements of practice that have been observed to contribute to tangible success in a variety of circumstances, in multiple industries and geographies.
  • 3. Contents
      • Research demonstrates a clear correlation between maturity and overall effectiveness – with some caveats 4
      • Overall, ACME is towards the lower quartile of the maturity spectrum compared to other companies surveyed 5
      • ACME overall delivery effectiveness appears to offer scope for improvement around some key outcomes 6
      • The most successful organizations demonstrate maturity across all key areas of practice 7
      • ACME’s practice maturity assessment suggests a framework for prioritization 8
      • Detailed analysis indicates specific areas for improvement 9
      • These may suggest an adaptation to ACME’s own assessment of improvement priorities 10
      • Diagnostic methodology 11
  • 4. Research demonstrates a clear correlation between maturity and overall effectiveness – with some caveats
    • An earlier PA survey into portfolio management practices and their resulting effectiveness confirmed that firms which systematically plan for success are more likely to achieve it. In other words, the more an organization puts in to managing its portfolio of projects, and the means of delivery, the more its gets out. While this was to be expected, there were a number of surprising findings:
      • Outliers, very large organizations with apparently mature processes, reported unsatisfactory outcomes
      • Similarly, outliers (generally smaller organizations) with ad-hoc processes, reported more favourable outcomes
      • In addition, while one might expect that the nature and scale of an investment programme (ie what is at stake) should drive the need for more mature processes, the survey indicated that this was not always the case. Certainly the most developed practitioners had substantial project portfolios, however, the two organizations with the largest portfolios had the least effective approaches.
    • Clearly certain organizational characteristics, relating to size, complexity and culture, are also significant factors.
    • This suggested, to us, the following categorization:
    • Entrepreneurial – Success characterized by the extraordinary efforts of talented individuals applying ad-hoc processes and tools – generally in smaller, more ‘agile’ organizations
    • Planned – Success was planned for by developing and inserting the right processes, technologies, and people.
    • Informal – Ad-hoc approach to portfolio development and management has not resulted in effective delivery.
    • Bureaucratic – Significant investment in standards, tools and processes is not supported by the necessary behaviours to deliver success.
    • Note:
      • Maturity is a measure of the quality of practices
      • Effectiveness is a measure of the quality of outcomes
      • Additional information is provided in the section on diagnostic methodology
    Source: PA Portfolio Management Survey Bureaucratic Entrepreneurial Maturity vs Effectiveness Maturity Effectiveness Planned Informal Most effective Least effective
  • 5. Overall, ACME is towards the lower quartile of the maturity spectrum compared to other companies surveyed
    • ACME has a medium sized portfolio of large in-house projects in support of the business and its customer base .
    • Approximate portfolio profile:
      • ~30 candidate projects per year (75% initiated)
      • Average value $2 MM
      • A mix of product development, IT, business change and asset acquisition projects
      • Serving multiple business units
      • Planning is geographically dispersed / decentralized.
    • ACME’s responses indicate that they are in the lower half of the maturity spectrum.
    The current survey in which ACME (and other organizations) have participated is more expansive than the original survey; the range of practices has been expanded beyond the confines of portfolio management into broader aspects of delivery. The results from both studies are consistent in the correlation of maturity and effectiveness - those companies that demonstrate more mature practices observe better outcomes. Source: PA Portfolio Delivery Effectiveness Survey, 2009 ACME
  • 6. ACME overall delivery effectiveness appears to offer scope for improvement around some key outcomes… Source: PA Portfolio Delivery Effectiveness Survey, 2009 The implications of shortcomings in other areas are potentially significant Portfolio balance appears to be in good order and is the best rated outcome today. Opportunities for improvement exist in all other areas. Portfolio outcomes Project outcomes Resourcing outcomes Sequential development, siloed working, protract and sub-optimize delivery, and impede multi-disciplinary efforts. Process – achieving efficiency/effectiveness in delivery Stakeholders have low expectations of delivery. Little or no delivery metrics captured or available. Stakeholder satisfaction – how well your delivery is perceived Accountability for delivery is unclear. It’s accepted that projects frequently run late or deliver short of original intentions. Accountability – taking ownership of delivery There is a tendency to start everything, but not deliver it all. Delivery reliability – what you achieve It is not clear how many projects are on the go, how they are tracking or what effort they absorb. Ability to adjust to changing priorities is limited. Portfolio adaptability – tracking and adapting the portfolio to current needs It is difficult to balance resources between business-as-usual and project work; resources are frequently overcommitted and often unilaterally redeployed. Allocation – matching demand and supply Business cases are unreliable, with poor estimating of project and business-as-usual demand, and little understanding of expected benefits. Demand estimating – understanding what you need Capability and capacity to deliver projects and business-as-usual is not well understood. Operational issues frequently disrupt change activities. Supply capacity – understanding what resources you have There is perpetually a shortage of skilled people; significant recourse to contract staff for operational and change activities. Utilization – understanding what you use Implications Dimension
  • 7. The most successful organizations demonstrate maturity across all key areas of practice
    • In comparing the performance of the most and least developed (mature) practitioners in the original survey, the best performers rated themselves consistently across the range of practices; in other words, they did everything reasonably well.
    • In contrast, the least mature rated themselves poorly in a number of fundamental areas. This indicates a number of significant discontinuities in what is essentially an end-to-end process – or perhaps a highly integrated meta-process – but which is not treated as such. Hence, overall effectiveness is vulnerable to the weakest link in the chain.
    • This is illustrated in the diagram.
    • The ‘most developed’ firms exhibited capability in all dimensions and were most differentiated by the:
      • Strength of linkages between projects and strategies (portfolio management)
      • Quality of their business cases (the front end of project management)
      • Proactive considerations of project constraints (resource and workforce management).
    • Some features initially thought to be essential appear less so:
      • Use of support systems (tools) to facilitate the process was found to be necessary, but not sufficient to improve effectiveness at most levels of maturity
      • Similarly, the presence of project management standards does not directly correlate with delivery effectiveness
      • The formality of the process and the prioritization technique applied were not found to significantly impact the effectiveness of project delivery.
    • At face value, some of these seem to be contradictory findings. However, at early stages of the maturity curve, technology may be largely irrelevant – more important are defining the strategies to guide project identification and the quality of project definition. At the mid point of the maturity curve where standardization and consistency is required, often technology takes the place of solid governance and often distracts the focus of work towards checklist completion, rather than excellent delivery. At the mature end of the curve, technology is a critical enabler for larger scale, complex environments (ie with multiple projects, or distributed teams and geographic dispersion of activities).
    • The issue of project management standards is interesting in that, while it is not possible to have highly effective processes without such standards, having them is no guarantee of delivery effectiveness. More important are the values, attitudes and behaviours of those responsible for delivery.
    Source: PA Portfolio Management Survey Strongly agree Ambivalent Strongly disagree Business objectives clear Strong links between objectives and selection Quality business cases Objective and credible selection process Scenarios rapidly evaluated Consider all significant constraints Flexible approach resilient to change Business impact of portfolio understood Efficient in use of management time Most effective Least effective
  • 8. ACME’s practice maturity assessment suggests a framework for prioritization
    • As overall effectiveness is vulnerable to discontinuities in the delivery chain, the first priority must be to improve those areas that impede delivery.
    • ACME’s results suggest that, from a practice maturity perspective, priority should be given to aspects of workforce and resource management – these are the areas of greatest “internal” discontinuity, and also represent the most significant variances with higher performing organizations.
    • The results also suggest that enhancements in portfolio and project management, workforce management (the strategic aspects of resourcing) also merit attention.
    • Elements of adaptive delivery and financial management are relative strengths. Improvements in these areas are least important and quite likely of limited value at this time.
    • .
    Source: PA Portfolio Delivery Effectiveness Survey, 2009 ACME
  • 9. Detailed analysis indicates specific areas for potential improvement
    • The resourcing challenge is the most severe and covers the strategic, tactical and operational aspects of resourcing (from capacity planning and acquiring staff and external resources, their organization, scheduling and allocation).
    • The portfolio management challenge covers initial prioritization and ongoing adaptation of the portfolio, as well as enablers such as the management information/dashboards to guide decision-making and clarity around accountability of each aspect of delivery.
    Those with a score less than this represent the greatest near-term improvement opportunities The portfolio management challenge The project management challenge The broader resourcing challenge
  • 10. These may suggest an adaptation to ACME’s own assessment of improvement priorities ACME’s own assessment of improvement priorities Alignment of suggested improvements
    • Resourcing enhancements (from strategic workforce aspects to operational scheduling and allocation) would raise delivery capacity and enable projects to be adequately staffed for delivery.
    • Improving ACME’s portfolio management (initial prioritization/selection and ongoing adaptation of the portfolio during the year) would improve the benefits potential of the portfolio itself – i.e. choosing better candidates.
    • Real benefits realization requires credible business cases followed by active planning for, tracking and holding to account those responsible for their achievement.
    • Addressing the program and project management challenges, (including such as knowledge management), together with the application of sufficient, capable staff to projects, would contribute to achieving these objectives.
    • In addition, consideration of those aspects that are often covered by a “design authority” (aspects of architecture, the design/development process, design assurance, as well as enablers such as the collaborative toolset) may also be warranted to achieve project efficiency and quality objectives.
    1= Enhancing the capacity to deliver more projects, without increasing cost Delivery capacity 1= Ensuring fitness for purpose and attaining a better result every time Delivery quality 1= Achieving tangible business results regularly and reliably throughout the planning horizon Benefits realization 1= Delivering results faster, more efficiently and hence at lower cost Delivery speed Priority Description Objective
  • 11. Diagnostic methodology
    • The diagnostic
    • The diagnostic measures respondents’ performance against key elements of practice that have been observed to contribute to tangible success in organizations in a variety of industries and geographies.
      • Questionnaire examines three areas:
        • Objectives: What you want
        • Practices: What you do
        • Outcomes: What you get
      • Practices and Outcomes are self-assessed against a 5-point scale 1 (poor/immature) through 5 (excellent/world class)
      • Objectives are self-assessed against a similar scale 1 (needs minor improvement) through 5 (needs major improvement)
    • This report
      • The report is intended to highlight areas for a respondent company to focus on to improve overall portfolio delivery performance
      • Practice and outcomes responses are consolidated into single Maturity and Effectiveness scores to compare relative overall performance of all respondents
      • The respondent’s practice scores are contrasted with the 90th, 75th, 25th and 10th percentiles of all survey respondents to highlight key areas of discontinuity driving differences in performance
      • The chart used to identify specific areas for improvement are predicted on the hypothesis is that major discontinuities in the maturity of a given practice represent the most serious bottlenecks to performance improvements – without addressing these first, any efforts in other areas are unlikely to improve outcomes appreciably.
      • This survey is intended as an initial, rapid diagnostic, based on the perceptions of respondents, to identify areas for potential improvement. The findings need to be followed up by substantive analysis to validate the findings and prioritize specific improvement initiatives.
  • 12. Contact details for further information concerning this report
    • For more information, contact:
    Are you interested to know how your organization’s project delivery performance compares with your peers? By participating in this research survey, you will have access to a complimentary benchmark report – like this one - that provides insight to how your organization compares to other respondents and areas to focus on. Your organization will be one of more than 90 in the US and Europe that have begun to use the report to make improvements on their performance. Here are some respondents’ comments on their reports: “ The survey clearly showed areas where our approach works well, but also gave specific areas for focus in improvement. The report is extremely helpful in reporting the PMO's progress as well as a tool for recommending improvement areas for the executive management.” “ This is unbelievably on the money for this group! As we’ve talked with internal and external stakeholders, these themes have consistently come up as major problems that they are facing.” Not a bad return for 20 minutes of your time? Please feel free to access the survey at: John Kirkwood One Memorial Drive Cambridge, MA 02142 Tel: +1 571 227 9351 Mobile: +1 202 320 0961 E-mail: [email_address] Tim Pare 4601 N Fairfax Drive Suite 600 Arlington, VA 22203 Tel: +1 571 227 9273 Mobile: +1 571 215 1332 E-mail: [email_address] John Hall 4601 N Fairfax Drive Suite 600 Arlington, VA 22203 Tel: +1 571 227 9351 Mobile: +1 202 320 0961 E-mail: [email_address]