Your SlideShare is downloading. ×
0
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
ERP success presentation in Shanghai, China, 1999
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

ERP success presentation in Shanghai, China, 1999

480

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
480
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Measuring the Effectiveness of ERP systems Peter Seddon, PhD Senior Lecturer Department of Information Systems The University of Melbourne [email_address] http://www.dis.unimelb.edu.au/staff/peter
  • 2. Measuring the Effectiveness of ERP systems Chinese version translated by Bin Hu Department of Information Systems The University of Melbourne [email_address]
  • 3. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 4. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 5. 1. Example calculation of ROI <ul><li>Report sponsored by SAP </li></ul><ul><li>Analysis of The Boston Beer Company’s investment in an SAP R/3 system </li></ul><ul><li>Implementation, January, 1996 </li></ul><ul><li>Standard IRR calculation: ROI of 83% </li></ul>Data from “The ROI Report”, Scalea and Company, 346 Beacon Street, Boston MA, August, 1997
  • 6. Boston Beer in 1997 <ul><li>Leading “craft” brewer in the US </li></ul><ul><li>contract brewer - using excess capacity of other brewers - plus one brewery in Cincinnati </li></ul><ul><li>380 employees: 190 in sales, 120 in brewing, and 70 in administration </li></ul><ul><li>1991-1996 net sales growth: 46% compound, from $30M to $191M </li></ul>
  • 7. Boston Beer’s SAP R/3 Modules <ul><li>FI Financials </li></ul><ul><li>SD Sales and Distribution </li></ul><ul><li>AM Asset Management </li></ul><ul><li>MM Material Management </li></ul><ul><li>CO Controlling </li></ul><ul><li>PP Production Planning </li></ul><ul><li>PA Profit Analysis </li></ul><ul><li>BS Basis (middleware) </li></ul>
  • 8. The Investment, 1995-1996
  • 9. Estimated Benefits, 1996-2000 <ul><li>Reduction in Gen. & Admin. Expense: 5% of $12.5m = $300,000 p.a. (rising to $900,000 by year 2000) </li></ul><ul><li>Interest Savings on A/R reduction: 6% of reduction of $4m in Working Cap = $240,000 (rising to $500,000 by year 2000) </li></ul><ul><li>Improved Budget & Control of Sales Expenses: $300,000 (rising to $800,000 by year 2000) </li></ul><ul><li>Improved Control of POS Expenses: $240,000 </li></ul><ul><li>Increased A/P Discounts: $100,000 p.a </li></ul>
  • 10.  
  • 11. Calculating ROI (=IRR), 1996?
  • 12. Boston Beer Share Price 1996-99 <ul><li>Source: http://www.etrade.com, 27 Nov 1999 </li></ul>
  • 13. Measuring ERP Effectiveness <ul><li>Boston Beer’s sales fell 3% during 1996-98. </li></ul><ul><li>Most of the expected cost savings (which assumed sales growth) were not realized. </li></ul><ul><li>The expected ROI was not achieved. </li></ul><ul><li>Does this mean the investment in the ERP system was a failure? </li></ul><ul><li>How should we measure ERP effectiveness? </li></ul>
  • 14. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 15. 2. Why is measuring IT Effectiveness so hard? ( not just for ERP systems) <ul><li>Often hard to link benefits to any specific IT investment (e.g., benefits of a LAN). </li></ul><ul><li>Cost savings are the easier to quantify, but these are not the only benefits. </li></ul><ul><li>There are many different goals for evaluation (e.g., feasibility, audit) </li></ul><ul><li>Different stakeholders evaluate the same system differently </li></ul><ul><li>Evaluations change over time </li></ul>
  • 16. Willcocks and Lester list six types of goals for IT investment <ul><li>Infrastructure building (can’t cost justify) </li></ul><ul><li>Cost efficiency (e.g., Boston Beer) </li></ul><ul><li>Service to the business (e.g., better information, better decision-making) </li></ul><ul><li>Enable business improvement (processes) </li></ul><ul><li>Differentiating the business competitively </li></ul><ul><li>Revenue generation </li></ul>Willcocks and Lester, Beyond the IT Productivity Paradox Wiley 1999
  • 17. Five Key Questions to get clear about IT Effectiveness evaluation <ul><li>Q1. From whose perspective is effectiveness being judged? </li></ul><ul><li>Q2. What is the system being evaluated? </li></ul><ul><li>Q3. What is the purpose of evaluation? </li></ul><ul><li>Q.4 What time frame is employed? (short, long) </li></ul><ul><li>Q.5 How is effectiveness to be judged? </li></ul>Based on: Seddon, Staples, Patnayakuni, and Bowtell, Dimensions of IS Success, Communications of the AIS October 1999
  • 18. Q1. From whose perspective is effectiveness being judged? <ul><li>independent stakeholder </li></ul><ul><li>individual user </li></ul><ul><li>a group of people </li></ul><ul><li>project manager </li></ul><ul><li>IT management </li></ul><ul><li>owner/senior management </li></ul><ul><li>society </li></ul>
  • 19. Q2. What is ‘the system’ being evaluated? <ul><li>part of an information system (e.g., GUI) </li></ul><ul><li>one particular system (e.g., the HR module) </li></ul><ul><li>a type of system (e.g., ERP) </li></ul><ul><li>all IT applications in an organization </li></ul><ul><li>an inter-organizational information system </li></ul><ul><li>an IT implementation process (e.g., ASAP) </li></ul><ul><li>the IT function in an organization </li></ul>
  • 20. Q.3 What is the purpose of the evaluation? <ul><li>Performance appraisal </li></ul><ul><li>to learn how to do better feasibility-study evaluations in future </li></ul><ul><li>expect different levels of cooperation from staff depending on purpose of evaluation </li></ul><ul><li>some evaluations may be much more comprehensive than others </li></ul>
  • 21. <ul><li>short term (e.g., “on time and within budget” for implementation projects) </li></ul><ul><li>long term </li></ul>Q.4 What is the time frame is employed?
  • 22. Q.5 How is effectiveness of the IT system to be judged? <ul><li>Compared to: </li></ul><ul><li>some other organization (benchmarking) </li></ul><ul><li>some ideal level of performance </li></ul><ul><li>stated goals of the organization (e.g., the feasibility study for an IT project) </li></ul><ul><li>past performance of the organization </li></ul><ul><li>other desirable characteristics </li></ul>
  • 23. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior managers: ERP effectiveness 6. Summary and Lessons
  • 24. Firms evaluate effectiveness at 3 points in the system’s lifecycle J.W. Ross, “The ERP revolution: Surviving versus Thriving,” Working Paper, CISR, MIT, 1998 Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time
  • 25. Firms evaluate effectiveness at 3 points in the system’s lifecycle J.W. Ross, “The ERP revolution: Surviving versus Thriving,” Working Paper, CISR, MIT, 1998 Feasibility Study Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time
  • 26. Firms evaluate effectiveness at 3 points in the system’s lifecycle J.W. Ross, “The ERP revolution: Surviving versus Thriving,” Working Paper, CISR, MIT, 1998 Feasibility Study Development Stage Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time
  • 27. Firms evaluate effectiveness at 3 points in the system’s lifecycle J.W. Ross, “The ERP revolution: Surviving versus Thriving,” Working Paper, CISR, MIT, 1998 Feasibility Study Development Stage Post-implementation Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time
  • 28. What percentage of firms evaluate their investments in IT? (formal reviews; not just ERP) Source: Seddon, Graeser, and Willcocks (1999): data from IT managers in 80 large European and US firms in 1998. Average annual turnover US$4.5B. Average annual IT budget was US$38 million.
  • 29. 1. Feasibility Stage Evaluation problems (survey of 97 UK firms, Ballantine 1993)
  • 30. 1. Feasibility Stage Evaluation Methods (80 respondents in US, UK, A & NZ, Bacon 1992)
  • 31. 2. Development-stage Evaluation Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time Development Stage
  • 32. 2. Development-stage Evaluation <ul><li>For project managers, the key success measure is often “ on time and within budget ”. </li></ul><ul><li>This has little to do with the economic value of the project. </li></ul><ul><li>Some projects are abandoned, which means the entire cost of the project is wasted. </li></ul><ul><li>Studying these abandoned projects gives insight into development-stage problems and evaluation </li></ul>
  • 33. 2. Development-stage Evaluation (Willcocks and Lester 1996, n=50) <ul><li>80% of the 50 organizations had abandoned projects because of negative development-stage evaluations. </li></ul><ul><li>The most important reasons for abandonment were: </li></ul><ul><li>project over budget (17/50 respondents) </li></ul><ul><li>organizational needs changed (23/50 respondents) </li></ul><ul><li>user requirements changed (22/50 respondents). </li></ul>
  • 34. 2. Development Stage Evaluation: Reasons for project abandonment (Norris 1996) <ul><li>Unacknowledged divisions between users on mandatory and desirable requirements and the scale of benefits expected from them. </li></ul><ul><li>Gung-ho attitudes to managing risks. </li></ul><ul><li>An assumption that short training courses at the launch of the system will be sufficient to change well-established working practices and to encourage users to adopt the system. </li></ul><ul><li>The above are consistent with findings on BPR. </li></ul>
  • 35. 3. Post-implementation Evaluation Post-implementation Perceived Organizational Performance Implementation Design Continuous Improvement Stabilisation Transformation Time
  • 36. 3. Criteria for Post-Implementation Review (Willcocks and Lester 1996, n=50) <ul><li>83% comparison to the feasibility study </li></ul><ul><li>63% cost-effectiveness </li></ul><ul><li>53% quality of product </li></ul><ul><li>48% systems availability </li></ul><ul><li>44% productivity </li></ul><ul><li>22% user satisfaction </li></ul><ul><li>15% of organizations used the top six criteria, and a further 15% used the top five (excluding user satisfaction). </li></ul>
  • 37. 3. Post-implementation Reviews Question: Source: Seddon, Graeser, and Willcocks (1999), n=80
  • 38. 3. Post-implementation Reviews Question: Source: Seddon, Graeser, and Willcocks (1999), n=80
  • 39. 3. Post-implementation Reviews The remainder of this presentation focuses on post-implementation benefits only, i.e., on benefits from use, once the system is operational.
  • 40. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 41. 4. A model of factors affecting individual perceptions of IS effectiveness (n=119;266) Information Quality Ease of Use Perceived Usefulness Perceived Net Benefit Importance of the task Knowledge of the system Support Staff 0.4 0.3 0.2 0.3 0.2 0.2 0.5 0.1 ns ns 0.2
  • 42. Comparing scores on Old and New Systems (September, 1996 and January, 1998. All scales: 1…7 ) Source: Seddon and Staples (1999)
  • 43. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 44. Why are senior managers interested in ERP effectiveness? <ul><li>To improve IT investment decision-making in future </li></ul><ul><li>To learn from reported experience of others </li></ul><ul><li>to understand capabilities of ERP systems and so tap possible sources of competitive advantage </li></ul><ul><li>to identify areas to target for future development </li></ul>
  • 45. Answers to the five key questions for evaluating IT effectiveness <ul><li>Stakeholder : senior management </li></ul><ul><li>System : the entire ERP system </li></ul><ul><li>Purpose of evaluation : previous slide </li></ul><ul><li>Time frame : costs and benefits to date, plus costs and benefits for the next 2-3 years (including software and hardware upgrade costs) </li></ul><ul><li>Effectiveness criteria : Balanced IT Scorecard </li></ul>
  • 46. Balanced IT Scorecard criteria <ul><li>1. From the corporate financial perspective, e.g., ROI </li></ul><ul><li>2. The customer/user perspective (e.g., on-time delivery rate, satisfaction) </li></ul><ul><li>3. Business process (e.g., purchase invoices per employee) </li></ul><ul><li>4. An innovation/learning perspective (e.g., rate of cost reduction for IT services) </li></ul><ul><li>5. From the systems project perspective (e.g., on-time, quality, cost) </li></ul><ul><li>6. A technical perspective (e.g., implementation efficiency, capacity utilization, response times) </li></ul>Source: Graeser and Willcocks (1998)
  • 47. Norris says for post-implementation evaluation of IT effectiveness, create a task force: <ul><li>Different stakeholder interests must be represented </li></ul><ul><li>Task force must have: </li></ul><ul><ul><li>Authority vested from the top </li></ul></ul><ul><ul><li>Board-level access </li></ul></ul><ul><ul><li>Company-wide emphasis </li></ul></ul><ul><ul><li>Business management insight. </li></ul></ul>Source: Norris (1996)
  • 48. Benefits checklist: Analysis of vendor success stories on WWW ( Shang, PhD in progress, 1999)
  • 49. Content analysis of the 182 WWW cases suggests that, from the perspective of senior management, there are five categories of benefits from ERP systems (Shang 1999): <ul><li>1. Operational </li></ul><ul><li>2. Management </li></ul><ul><li>3. Strategic </li></ul><ul><li>4. IT infrastructure </li></ul><ul><li>5. Organizational </li></ul>
  • 50. 1.Operational benefits (Shang 1999) <ul><li>Cost reduction : labour, training, inventory </li></ul><ul><li>Cycle-time reduction : </li></ul><ul><ul><li>faster delivery to customer, </li></ul></ul><ul><ul><li>faster administrative processes </li></ul></ul><ul><li>Increased productivity </li></ul><ul><li>Improved data quality </li></ul><ul><li>Improved customer service </li></ul>
  • 51. <ul><li>Better resource management : inventory, maintenance, production scheduling, workforce management </li></ul><ul><li>Better decision making : improved market responsiveness, fast response to work changes, fast response to customer needs </li></ul><ul><li>Better control : analysis by line of business, product, customer, geographic area; production costs management </li></ul>2.Managerial benefits (Shang 1999)
  • 52. <ul><li>Support for future business growth </li></ul><ul><li>Support for business alliances </li></ul><ul><li>“ Lock-in” customers (realtime data sharing, interactive customer service) </li></ul><ul><li>build business innovation </li></ul><ul><li>build cost leadership </li></ul><ul><li>enhance product differentiation </li></ul><ul><li>sustain competitiveness </li></ul>3.Strategic benefits (Shang 1999)
  • 53. <ul><li>Increased business flexibility </li></ul><ul><li>reduction in IT costs : legacy systems maintenance, mainframe replacement, year 2000 compliance </li></ul><ul><li>increased infrastructure capability : global platform, database integrity </li></ul><ul><li>flexibility : adaptable modern technology, extendable, compatible </li></ul>4.Infrastructure benefits (Shang1999)
  • 54. <ul><li>Support organizational changes (restructuring) </li></ul><ul><li>business and employee skills learning </li></ul><ul><li>greater customer focus for staff </li></ul><ul><li>empowerment and accountability </li></ul><ul><li>teamwork </li></ul><ul><li>better employee morale and satisfaction </li></ul>5.Organizational benefits (Shang1999)
  • 55. <ul><li>Vendor success stories do not mention the problems of ERP system use such as: </li></ul><ul><li>Dependence on vendor upgrades </li></ul><ul><li>Poor response times </li></ul><ul><li>Inflexibility/Expensive customization </li></ul><ul><li>Staff leaving for better salaries </li></ul><ul><li>others </li></ul>Downsides of ERP systems
  • 56. <ul><li>Persuade a number of user organizations to use a combination of the IT Scorecard and the ERP Benefits Checklist to assess the effectiveness of their current systems. </li></ul><ul><li>Will also ask them to identify downsides (unwanted consequences) of ERP systems. </li></ul><ul><li>Hope to identify some common factors that led to greater perceived success . </li></ul><ul><li>Expect user knowledge to be a key determinant of success. </li></ul>Plan (for next year)
  • 57. Measuring the Effectiveness of ERP systems <ul><li>1. An example calculation of ROI </li></ul><ul><li>2.Why is measuring IT Effectiveness so hard? </li></ul>3.What do we know about IT Effectiveness measurement in practice? 4. Individuals: IT effectiveness 5. Senior Managers: ERP effectiveness 6. Summary and Lessons
  • 58. 6. Summary and Lessons <ul><li>IS effectiveness measurement is difficult because there is no simple causal relationship between IT expenditure and benefits. </li></ul><ul><li>Financial measures such as Boston Beer’s ROI, and changes in corporate profitability, are clearly invalid in some circumstances. </li></ul><ul><li>The previous section contains our best recipe for ERP effectiveness measurement from the senior management perspective: </li></ul><ul><li>Taskforce + scorecard + checklist. </li></ul>
  • 59. Finally: <ul><li>“ Establishing value for money depends on the business judgement of the managers involved - it is no more amenable to numerical analysis than any other value judgement.” (Norris 1996) </li></ul>Source: Norris, G.D. Post-investment appraisal, in Willcocks L. Investing in Information Systems , London: Chapman and Hall, 1996: 193-221.
  • 60. Questions? <ul><li>Peter Seddon and Bin Hu </li></ul><ul><li>Department of Information Systems </li></ul><ul><li>The University of Melbourne </li></ul><ul><li>[email_address] </li></ul><ul><li>http://www.dis.unimelb.edu.au/staff/peter </li></ul>

×