Executive Summary


Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Executive Summary

  1. 1. Review of Construction Key Performance Indicators [BERR reference ITT No 4] Final Report Submitted by Manchester Business School University of Manchester United Kingdom 25th April 2008 Manchester Business School University of Manchester Oxford Rd Manchester M13 9PL Tel: +44 (0)161 275 5928
  2. 2. Review of UK Construction KPIs – Final Report Executive Summary This report to the Department of Business, Enterprise and Regulatory Reform (BERR) is the outcome of a Review of the UK Construction Key Performance Indicators. The Review commenced in December 2007 and was completed in April 2008. An Interim Report was submitted to BERR on 29th February 2008. Background and objectives of the Review (Chapter 1) The UK Construction Key Performance Indicators (referred to subsequently as the KPIs) were first published in 1999 as a means of monitoring the overall performance of the construction sector and its progress towards Rethinking Construction targets. The current contract for their preparation dates from 2002 and was due to end in 2008, but has been extended to 2009. The Review was commissioned by BERR to inform decisions over future Government support for the production of the KPIs. Its objectives (specification in Annex A) included:  Examining whether the KPIs meet the objectives set for them, and the benefits derived from their publication.  Assessing the extent and distribution of the resource inputs required to produce them.  Examining the case for their continuation, and for future government support, and presenting these in a Business Case for the KPIs.  Providing an account of the present arrangements and an inventory of outputs.  Making recommendations for changes in the content of the KPIs and the processes involved in data collection and KPI preparation. Review process (Chapter 1) The Review process consisted of: 1) Examination of reports and other literature relevant to the KPIs, notably those that followed the publication of Rethinking Construction. 2) Interviews with members of the ‘KPI Consortium’ who currently collect data for and prepare the KPIs. 3) Preparation of an interview structure and on-line questionnaire to gain industry views, followed by the conduct of the interviews and promotion of the questionnaire. 4) International enquiries to gain different perspectives on construction benchmarking. 5) Analysis of the information gained and preparation of the report. Background and history of the KPIs (Chapter 2) This chapter presents a review of the genesis and development of the KPIs, as recorded in various official reports, with relevant extracts from the reports. The principal conclusion is that the KPIs developed in a way that differed from the original proposal in Rethinking Construction, which envisaged more focus on their use by clients and the publication of performance data for individual firms. Instead, the main function of the KPIs has been to provide distributions of different aspects of industry performance with which firms can compare their own performance. This has reduced the need for full statistical reliability in the outputs and consequently reduced the costs of preparation. Present arrangements for producing and publishing the KPIs (Chapter 3) The KPIs are produced by a consortium of industry bodies led by BPA Consult. The data are collected from a range of sources, including national statistics, but key elements of the data are obtained from specially commissioned surveys, for which responses are voluntary. These are conducted through the distribution of paper-based survey forms, on-line survey processes and telephone surveys. The distributions of respondents are compared with the distributions 2
  3. 3. Review of UK Construction KPIs – Final Report of industry enterprises and outputs known from statutory returns and measures taken to provide final outputs that better reflect the industry distributions. Publication of the KPIs by Construction Excellence (CE) is Web-based (www.kpizone.com) with the detailed outputs available only on subscription. The site includes facilities for users to input and store their own data and to make comparisons with their choice of KPIs. Prior to 2007, paper-based outputs were available. Annex B presents an inventory of KPI-related outputs and data on the distribution of printed publications prior to 2007. Annex C presents a breakdown of the annual costs of preparing and publishing the KPIs, including estimates of the value of industry ‘in- kind’ contributions. The total cost is around £1m, of which government contributes around a half. Results of enquiries (Chapter 4) Analyses of the information obtained from the interviews and questionnaire are presented. Annex D reproduces the guidance prepared for interviewers and Annex E lists the interviewees. Annex F presents the text of the on-line questionnaire. 32 interviews were conducted and 129 usable responses were received to the questionnaire. The latter figure was too low to permit analyses of responses by individual sectors of construction. The questionnaire respondents were not (and were not expected to be) representative of construction generally, but provided a wider set of perspectives on the KPIs. The interviewees were current or previous users of the KPIs. Their overall view of the KPIs was positive but most had combined use of the KPIs with indicators devised to suit their particular circumstances. Several themes emerged from interview responses: • There was strong support for formal measurement systems as a tool in performance improvement, whether or not the KPIs were used. • While the KPIs provided a suitable basis for performance monitoring within the organisation, doubts over the comparability of data inhibited a significant proportion of the interviewees from using the national distributions to benchmark themselves against other organisations. • Many organisations had used data from the KPIs or other indicators in marketing on in response to tender invitations. • Some organisations had commenced performance measurement using the KPIs but had then preferred to use their own indicators. • Their use in procurement was confined to public sector bodies and social housing providers. Around two-thirds of the questionnaire respondents engaged in performance measurement and benchmarking with 55 stating that they used the KPIs in for these purposes. Around half of these used the KPIs in supplier selection and management. The responses suggest a high level of support for the KPIs with 35 respondents considering them ’significant’ or ‘highly significant’ in the decision to take improvement actions. Few respondents were prepared to put a monetary value on the benefits from use of the KPIs but several of those who did indicated a figure of over £500k. From the questionnaire also: • Organisations used a wide range of information when deciding on improvement actions; there seemed fewer (but still some) reservations about making comparisons with the national KPI distributions, but Demonstration Project data appeared not to be widely used. • Most organisations using the KPIs had attended a workshop or Masterclass. • There were some calls for the KPIs to be made available without charge, but this did not appear to be a widespread view. • There were proposals for additional indicators relating to environmental performance and to project team relationships. 3
  4. 4. Review of UK Construction KPIs – Final Report The information obtained on construction benchmarking schemes in other countries showed that the most detailed scheme to be that in Denmark, where the Benchmarking Centre for the Danish Construction Sector operates a system aimed at providing data on contractor and consultant performance for public sector clients, which they are required to utilise. This has independent collection and verification of data, but is much more expensive to operate than the KPI system. By contrast, in South Africa national data for construction benchmarking are largely based on subjective evaluations supplied in voluntary response to surveys. The newly- introduced Scottish system also, at the entry level, relies on voluntary submission of subjective data. The effectiveness and impact of the present KPIs (Chapter 5) The present KPIs were reviewed against the objectives as stated in the contract for their preparation. The overall conclusion is that they meet those objectives, although the distributions may have a bias towards presenting an optimistic picture of industry performance and the Review Team have some reservations about the reliability of the year-to-year trends, particularly for indicators that depend on individuals’ perceptions. The KPIs were also reviewed against the objectives as stated in Rethinking Construction and with the three generic objectives for construction benchmarking: • Stimulating and informing improvement actions by firms. • Informing the markets for construction services. • Monitoring aspects of performance of the industry. They do not address several of the objectives put forward in Rethinking Construction but meet the generic objectives, with some reservations about their ability to inform construction markets. The objectives set for the KPIs relate to their preparation and not to the ultimate impact on industry practice. The Review Team attempted to assess this impact, but in the absence of reliable data on the extent of usage and of the level of benefits gained no figure can be provided on the scale of benefits. While several thousand organisations have participated in KPI Masterclasses etc, these still represent only a small proportion of the firms and client bodies of the industry. And the number of subscribers to KPIzone, or who have access through membership of Constructing Excellence, is much smaller. However, these attendees and subscribers include many large organisations; the KPIs would need to have only modest impact to result in benefits that far exceeded the cost of their production The principal direct beneficiaries appear to be the firms that draw on the KPIs to stimulate and guide improvement actions. Indirectly, clients benefit from this, and public sector clients in particular may also be direct beneficiaries through use of the KPIs in procurement actions. BERR Construction Sector Unit confirmed that they regarded the KPIs as a vital source of information for policy purposes but this could not be valued in financial terms. Thus a comparison of the distribution of costs with the distribution of benefits indicates that, if the intention is to achieve greater alignment of costs and benefits, users of KPIs should contribute more to the cost of their preparation. Future needs and options (Chapter 6) This chapter first discusses the case for the continuation of a system of construction benchmarking and then the case for continued Government support, concluding with a review of options for the type of KPI system and possible changes to improve its effectiveness. The classical case for having a system of performance indicators, in terms of its contribution to a more effective market for construction services, is set out; this is supported by a review in Annex H of the academic literature on the role of benchmarking in promoting improved industrial performance. The conclusion is that while the KPIs as they have developed have perhaps not had as much market influence as was originally envisaged, the characteristics of construction still support a case for their preparation. 4
  5. 5. Review of UK Construction KPIs – Final Report The policy context for any future government funding is then considered, with the conclusion that Government has a need for monitoring the performance of the industry, particularly as it develops measures to promote sustainable construction – including not only environmental performance but also the operation of efficient, profitable firms – and sustainable procurement. In view of this, there is a case for continued government funding, but with most of the benefits for the pubic sector coming through the impact on construction industry performance and therefore obtained by clients, the proportion of ‘central’ funding might decline over the period of the next contract, with efforts made to promote use (and therefore subscriptions) by public sector clients and their suppliers, thus distributing the public sector contribution over a larger number of bodies. Options for the future type of KPI system are considered, with a conclusion that the present approach should continue but with an attempt to increase the direct market-relevance of the indicators though the introduction of a degree of independent verification and more attention paid to issues of take-up and impact. In addition, greater transparency in the processes that lead to the publication of the KPIs would address some of the reservations expressed by interviewees over their use of external benchmarking and those of the Review Team over the reliability of year-to-year trends. This is consistent with the review of the implications of Freedom of Information (FOI) and Data Protection legislation presented in Annex G; amongst its conclusions are that compliance with FOI legislation for the KPIs may well result in the provision of additional information on the original data and on analysis processes. Other proposals include:  Creation of a high-level industry steering group in order to secure more evident industry ‘ownership’ of the KPIs.  Reviewing the content of the Environmental KPIs and possibly extending the Respect for People KPIs.  Keeping on-line data collection under review and extending its usage once Web usage in the industry rises. Summing up - the overall Business Case for the KPIs and findings from the Review (Chapter 7) Because of the paucity of data over impacts and financial benefits, it has not proved possible to assemble a formal Business Case for the KPIs. But a case for their continuation, and for continued support from central government funds, can be made in relation to each of the three generic objectives stated for construction benchmarking. And there is evidence that they have been a factor in stimulating improvement actions by firms. Overall, therefore, there is a case for the continuation of the KPIs, even if this cannot be expressed in quantitative terms. The findings of the review may be summarised as: 1) The present system of UK Construction KPIs has evolved in a way that differs somewhat from the original objectives set out in Rethinking Construction, in particular in placing much less emphasis on the role of the KPIs in informing the various markets for construction services. These differences are formalised in the objectives for the KPIs set out in the present contract for their preparation. 2) With some caveats, notably on the lack of information about the uncertainties in the data presented, the KPIs meet those contractual objectives. Hence they:  Offer firms and other bodies a means of measuring their performance and comparing it with that of others.  Provide a basis for comparing the performance of selected groups of projects such as the Demonstration Projects with that of the industry as a whole.  Give an indication of the progress made by the industry towards Rethinking Construction targets. 5
  6. 6. Review of UK Construction KPIs – Final Report 3) Making estimates for the value of ‘in-kind’ contributions, the preparation and promotion of the KPIs costs around £1 million each year, of which around half is borne by the Government. 4) We are unable to arrive at a meaningful figure for the value of benefits derived from the KPIs but it is reasonable to conclude that these exceed the costs by a considerable margin. 5) The benefits appear to accrue in the first place principally to firms taking improvement actions (and therefore indirectly to clients) but KPIs are also used by clients, particularly in the public sector, and these therefore are direct beneficiaries. 6) The present KPI system represents a compromise between a high-cost, highly validated approach and a very low cost system which would not provide the ability to monitor industry performance. It offers a suitable base for future development, but its limitations have to be recognised. 7) There is a case for continued government support for the KPIs on each of the three generic grounds for benchmarking in construction;  Stimulating and informing improvement actions by firms.  Informing markets and therefore procurement actions.  Monitoring performance at industry and sector level. In relation to the last, Government has a need to monitor the impact of measures taken to promote sustainable construction and sustainable procurement and the KPIs can play a role in this. The monitoring arrangements for these policies should be co-ordinated. 8) If the aim is to align more closely the distribution of financial support with the distribution of benefits, direct Government funding should diminish – not immediately, but over the period of the next contract. In parallel, Government should use its influence with public sector clients to encourage use of the KPIs by them and their suppliers. 9) If this course of action is adopted, the objectives set out for the KPIs in future should reflect this intention that they be used to inform construction markets, and might also make specific reference to their use in the monitoring of sustainable construction and sustainable procurement. 10) To address concerns over the reliability of the national distributions provided by the KPIs, there should be greater transparency in KPI processes and more information on the level of uncertainty in the graphs. Compliance with Freedom of Information legislation may in any case lead to the publication of this information. 11) Further, BERR should review procedures and the assurances given on the handling of personal data, to ensure compliance with the Data Protection Act 1998. 12) The KPIs should be more evidently ‘owned’ by the industry, with a new industry group being established to oversee the development of the system. 13) Greater attention should be paid to the level of take-up and the impact of the KPIs, as part of the move towards a greater share of funding coming from users, with the introduction of more systematic surveys of usage. 14) Use of on-line data collection processes should be kept under review. 6
  7. 7. Review of UK Construction KPIs – Final Report 15) The Environmental KPIs should be reviewed to ensure that they adequately reflect the need for construction projects to exhibit a high standard of environmental performance and the Respect for People KPIs might be enhanced to reflect the social dimensions of sustainability. 16) Use of the KPIs in the course of projects and in monitoring longer-term relationships might be promoted, with the possible introduction of additional indicators for monitoring the quality of relationships. 7