Challenges in Business Performance Measurement: The Case of a ...


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Challenges in Business Performance Measurement: The Case of a ...

  1. 1. Challenges in Business Performance Measurement: The Case of a Corporate IT Function Stephen Corea & Andy Watters (Warwick University, UK)
  2. 2. Presentation Outline <ul><li>Research Motivation </li></ul><ul><li>Theoretical Review </li></ul><ul><li>Methodology </li></ul><ul><li>Case Study: GITS </li></ul><ul><li>Findings & Discussion </li></ul><ul><li>Conclusion </li></ul>
  3. 3. <ul><li>Business performance measurement (PM) – presenting relevant information to management staff for assessing the organization's progress towards achieving strategic/operational aims </li></ul><ul><ul><li>Several major PM frameworks proposed recently: field dominated by prescriptive, top-down perspective (formal derivation from strategy) </li></ul></ul>Research Background <ul><li>RESEARCH AIM : </li></ul><ul><li>exploratory study to understand the challenges in mounting dashboard based PM practices in a corporate IS Dept </li></ul><ul><ul><li>Significant no. of PM initiatives fail to take root or adequately deliver expected benefits (70% according to McCunn, 1998) </li></ul></ul><ul><ul><li>Need for greater understanding of on-the-ground challenges and bottom-up perspective of challenges involved </li></ul></ul>
  4. 4. <ul><li>Robust PM system should take a ‘balanced’ approach (Kaplan & Norton, 1992) </li></ul><ul><li>Identification/population of useful performance measures (or metrics) to capture progress towards goal attainment is a key but not easily satisfied criteria (Neely et al, 1997): </li></ul>Theory: PM Implementation <ul><li>Key proposed principles of PM design </li></ul><ul><ul><li>Should be grounded in strategy: performance metrics must be derive from predefined strategy/objectives (De Tonia & Tonchia, 2001) </li></ul></ul><ul><ul><li>Should incorporate a high proportion of ‘leading’ indicators (Eckerson, 2006) i.e. in comparison to ‘lag indicators’ </li></ul></ul>
  5. 5. Theory: PM Design <ul><li>22 most-cited recommendations for designing measures (Neely e al, 1997) </li></ul><ul><li>Provide fast feedback </li></ul><ul><li>Have an explicit purpose </li></ul><ul><li>Be based on explicitly defined formula and source of data </li></ul><ul><li>Employ ratios rather than absolute numbers </li></ul><ul><li>Use data which are automatically collected as part of a process </li></ul><ul><li>Be reported in a simple consistent format </li></ul><ul><li>Be based on trends rather than snapshots </li></ul><ul><li>Provide information </li></ul><ul><li>Be precise – be exact about what is being measured </li></ul><ul><li>Be objective – not based on opinion </li></ul><ul><li>Be derived from strategy </li></ul><ul><li>Be simple to understand </li></ul><ul><li>Provide timely and accurate feedback </li></ul><ul><li>Be actionable: based on quantities that can be influenced or controlled </li></ul><ul><li>Reflect the business process: both customer and supplier should be involved in defining </li></ul><ul><li>Relate to specific targets </li></ul><ul><li>Be relevant </li></ul><ul><li>Be predictive: part of a closed management loop </li></ul><ul><li>Be clearly defined </li></ul><ul><li>Have visual impact </li></ul><ul><li>Should focus on improvement </li></ul><ul><li>Be consistent (i.e. should maintain significance as time goes by) </li></ul>
  6. 6. <ul><li>Systemic aspect of effective PM: process rationalisation, shared understanding & staff commitment, IT support (data capture/collection, processing & presentation) </li></ul><ul><li>Dashboards: visual impact, data quality and timeliness (Few, 2005; Dixon et al., 1999) </li></ul><ul><li>PM for the IT Function: need for spread of measures across 3 categories (Stanwick & Stanwick, 2005) – (i) efficiency; (ii) effectiveness; (iii) productivity </li></ul>Theory: PM Implementation
  7. 7. <ul><li>Interpretive case study method (Walsham, 1995): indepth single-site case study, aimed at theoretical generalisation </li></ul><ul><li>Case organisation/unit: GITS (Group IT Services), the corporate IT function of Multicorp (a pseudonym), a multi-national manufacturer of tobacco-based products </li></ul><ul><li>Data Gathering & Analysis </li></ul><ul><ul><li>multiple site visits: June to August 2006 </li></ul></ul><ul><ul><li>27 semi-structured interviews, dashboards/documents review, informal conversations </li></ul></ul><ul><ul><li>inductive analysis (Glaser & Strauss, 1967): identifying patterned regularities (common themes, issues or dilemmas) </li></ul></ul>Research Method
  8. 8. Case Study: Multicorp /GITS <ul><li>GITS (Groups IT Services) </li></ul><ul><ul><li>corporate IT function, formed in 2004 </li></ul></ul><ul><ul><li>transform supply of IT support across Multicorp’s worldwide business units from traditional geographically-localised model into a centralised, ‘shared services’ model </li></ul></ul><ul><ul><li>to achieve cost-savings of £100 million by 2009 </li></ul></ul><ul><li>Major manufacturer of cigarettes & tobacco-based products </li></ul><ul><ul><li>300+ brands sold in 180 ‘end-markets’ (i.e. country-specific regions); factories in 54 countries </li></ul></ul><ul><ul><li>stable industry competitive conditions: strategy is strongly focused on efficiencies (cost-savings) across business units (esp. support) </li></ul></ul>
  9. 9. Case Study: GITS <ul><li>Strategy & structure </li></ul><ul><ul><li>vision: irresistible value </li></ul></ul><ul><ul><li>client-facing units & management team </li></ul></ul><ul><ul><li>critical need to monitor performance </li></ul></ul><ul><ul><li>3 dashboards in use </li></ul></ul><ul><ul><li>“ Two years ago GITS had less than 100 staff, the management team could sit in a room and discuss in detail operational issues throughout the department. Now we’re more than 500 strong, and are doing far more things. We haven’t a clue what is going on out there, and don’t know what operational things we should be looking at.” (Manager) </li></ul></ul>World Class People Service Quality Irresistible Value Cost Savings SDS 26 Volume £ 100m CSS 4.5 100% Supply Side
  10. 10. GITS: Leadership Dashboard
  11. 11. GITS: Application Services Dashboard
  12. 12. GITS: Technical Services Dashboard
  13. 13. <ul><li>Inadequacies in dashboard population and scope of measurement </li></ul><ul><ul><li>difficulties obtaining timely & accurate data </li></ul></ul><ul><ul><li>areas of performance left untracked (in scope & time) </li></ul></ul><ul><ul><li>deficient in leading indicators (heavily lagged-oriented): lack of predictive capacity to take proactive interventions </li></ul></ul><ul><ul><ul><li>““ I’ve no idea what drives the numbers. I’m not sure if anyone has” (manager) </li></ul></ul></ul>Case: Findings
  14. 14. <ul><li>lack of clarity or common understanding regarding definition of certain measures, e.g. </li></ul><ul><ul><li>Constitution of measures: e.g. managed volume </li></ul></ul><ul><ul><ul><li>(i) “We count managed volume against our target only when services have been transferred to GITS, and the first invoice sent to the end-market”; </li></ul></ul></ul><ul><ul><ul><li>(ii) “Managed volume is just that: services which we (GITS) manage. It doesn’t matter if we haven’t billed the customer yet.” </li></ul></ul></ul><ul><ul><li>Progress towards targets: e.g. cost savings </li></ul></ul><ul><ul><ul><li>(i) “We claim that we have achieved a cost saving when we sign a contract with an outsource provider to provide the service at a cost lower next year than our current deal” </li></ul></ul></ul><ul><ul><ul><li>(ii) “Cost savings are claimed when we release next years’ price list to the end markets in May, with confirmation in early December.” </li></ul></ul></ul>Case: Findings
  15. 15. <ul><li>Relation of measurement to strategy: difference between Leadership & AS/TS dashboards </li></ul><ul><ul><li>Application Services & Technical Services dashboards reported self-chosen operational targets beyond existing strategy </li></ul></ul>Case: Findings <ul><ul><li>non-strategy derived measures were seen by managers as useful aspects for monitoring operational health </li></ul></ul><ul><ul><ul><li>“ There are quite a few measures which don’t directly relate to strategy or targets, but we think it is worthwhile to keep track of them. It helps to know the operational health of the business.” (Application Services manager) </li></ul></ul></ul><ul><ul><li>political value or ‘signaling’ </li></ul></ul><ul><ul><ul><li>““ We benchmark the charge rates of our project managers against external consultancy providers; we’re less expensive, and the difference is classed as Cost Avoidance. It helps us demonstrate our value to the business.” (Technical Services manager) </li></ul></ul></ul>
  16. 16. <ul><li>Lack of systemisation in data collection and measurement (cum dashboard) design </li></ul><ul><ul><li>no top-down mandate or formal programme / framework guiding the implementation of these practices: need for process rationalisation and information systems infrastructure (Bourne et. al, 2003) </li></ul></ul><ul><ul><li>difficulty identifying leading indicators (Neely et al., 2000; Eckerson, 2006) </li></ul></ul>Case: Discussion <ul><li>Need for re-orientation of fundamental aim/focus </li></ul><ul><ul><li>from a tool for simply monitoring/reporting to one of learning what factors drive results (i.e. to be able to influence/control) </li></ul></ul>
  17. 17. <ul><li>Re-thinking major PM tenets/principles </li></ul><ul><ul><li>Notion of ‘balance’ in balanced measurement </li></ul></ul><ul><ul><ul><li>financial vs. non-financial ‘lever’ </li></ul></ul></ul><ul><ul><ul><li>reporting vs. predication/learning ‘lever’ (lagging vs. leading indicators) </li></ul></ul></ul>Case: Discussion <ul><ul><li>Existing strategy as the source for deriving measures: a case for de-coupling strategy from measurement? </li></ul></ul><ul><ul><ul><li>cost-focussed strategies promote financial and discourage non-financial indicators </li></ul></ul></ul><ul><ul><ul><li>‘ de-politicizing’ of measurement </li></ul></ul></ul><ul><ul><ul><li>promote transformation of PM towards a learning rather than simply a monitoring tool </li></ul></ul></ul>
  18. 18. <ul><li>Results of this exploratory study suggest a need for further research & theoretical development to extend & deepen understanding of the complex nature of PM </li></ul><ul><ul><li>What does ‘balanced’ measurement imply </li></ul></ul><ul><ul><li>Relationship between strategy and measurement </li></ul></ul><ul><li>Questions? </li></ul><ul><li>Thank you </li></ul>Conclusion
  20. 20. References <ul><li>Kaplan, R. and Norton, D. (1992). “The balanced scorecard – measures that drive performance.” Harvard Business Review. January-February, 71-29. </li></ul><ul><li>McCunn, P. (1998) The Balanced Scorecard: the eleventh commandment. Management Accounting 34-36. </li></ul><ul><li>Neely, A., Richards, H., Mills, J., Platts, K. and Bourne, M. (1997) Designing performance measures: a structured approach. International Journal of </li></ul><ul><li>Operations & Production Management 17, 1131-1152. </li></ul><ul><li>De Toni, A. and Tonchia, S. (2001) Performance measurement systems - models, characteristics and measures. International Journal of Productions and Operations Management 1, 347-354. </li></ul><ul><li>Eckerson, W. (2006) Performance Dashboards: Measuring Monitoring and Managing Your Business, edn. New Jersey: John Wiley & Sons. </li></ul><ul><li>Few, S. (2005) Dashboard Design: Beyond Meters, Gauges, ad Traffic Lights. Business Intelligence Journal 10, 18-24. </li></ul><ul><li>Dixon, J.R., Nanni, A.J. and Vollmann, T.E. (1990) The new performance challenge: Measuring operations for world-class competition, Homewood, IL: Dow Jones-Irwin. </li></ul><ul><li>Stanwick, P. and Stanwick, S. (2005) IT Performance: How Do You Measure a Moving Target? The Journal of Corporate Accounting and Finance 13, 19-24. </li></ul><ul><li>Walsham, G. (1995). Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4:2, 74-81. </li></ul><ul><li>Glaser, B. and Strauss, A. (1967) The discovery of grounded theory, Aldine, Chicago. </li></ul><ul><li>Bourne, M., Franco, M. and Wilkes, J. (2003) Corporate Performance Management. Measuring Business Excellence 7, 15-21. </li></ul>