Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluating and selecting software packages a review

4,048 views

Published on

Published in: Technology, Education
  • Be the first to comment

Evaluating and selecting software packages a review

  1. 1. Evaluating and selecting software packages: A review Anil S. Jadhav*, Rajendra M. Sonar Information and Software Technology vol.51 pp.555–563, 2009. Big Wu Presenter : Jin Liu Dean Yeh
  2. 2. Keyword • Software Evaluation • Software Selection • Evaluation Criteria • Software Selection Tools 2
  3. 3. Overview • Evaluating and selecting software packages • That meet an organization’s requirements is a difficult software engineering process. • The aim of this paper is to provide a basis to improve the process • Selection of a wrong software package • Can turn out to be costly and adversely affect business processes. 3
  4. 4. Overview • This paper reports a systematic review of papers published in journals and conference proceedings. • The review investigates methodologies that support decision makers include: Selecting Software Packages Software Evaluation Techniques Software Evaluation Criteria System 4
  5. 5. Overview • The key findings of the review are: 1. Analytic Hierarchy Process(AHP) has been widely used for evaluation of the software packages. 2. Lack of a common list of generic software evaluation criteria and its meaning. 3. Need to develop a framework comprising to assist decision makers in software selection. 5
  6. 6. Introduction • B. Hecht (1997) • Selecting the right solution is an exhausting process for companies. • J. Verville, A. Hallingten (2002) • As ERP packages cost hundreds of thousands and even millions of dollars, purchasing an ERP solution is a high expenditure activity that consumes a significant portion of companies’ capital budgets. 6
  7. 7. Research Questions RQ1: • What is the contribution of the literature in the field of evaluation and selection of the software packages? RQ2: • What are the methodologies for selecting software packages? RQ3: • What are the systems/tools to assist decision makers in evaluating and selecting software packages? RQ4: • What are the software evaluation techniques? RQ5: • What are the software evaluation criteria? 7
  8. 8. Multiple Criteria Decision Making (MCDM) • K. Yoon, C. Hwang (1995) & E. Triantaphyllou (2000) • MCDM refers to making preference decisions over the available alternatives that are characterized by multiple, usually conflicting, attributes. • M. Mollaghasemi, J. Pet-Edwards (1997) • The goal of the MCDM is to help 1. Decision makers choose the best alternative of those studied. 2. Dort out alternatives that seem good among the set of alternatives studied. 3. Rank the alternatives in decreasing order of performance. 8
  9. 9. Literature review P.K. Lawlis, K.E. Mark, D.A. Thomas, T. Courtheyn, (1997) B. Kizzort, (2001) • A formal process for evaluating COTS software products, IEEE Proceedings Software • Selection of components for OTS component based systems, IEEE S. Comella-Dorda, J.C. Dean, E. Morris, P. Oberndorf, (2002) • A Process for COTS Software Product Evaluation, ICCBSS K.R.P.H. Leung, Hareton K.N. Leung, (2002) • On the efficiency of domain-based COTS product selection method, Information and Software Technology C.-D. Santiago, C. John, M. Edwin, O. Patricia, (2002) • A process for COTS software product evaluation, ICCBSS 9
  10. 10. Literature review D. Kunda, (2003) K. Oh, N. Lee, S. Rhew, (2003) • STACE: Social Technical Approach to COTS Software Evaluation Component Based Software Quality, LNCS, • A selection process of COTS components based on the quality of the software in a special attention to internet, HIS G. Grau, J. Pablo Carvallo, X. Franch, C. Quer, (2004) • DesCOTS: a software system for selecting COTS components, in: Proceedings of the 30th EUROMICRO Conference, IEEE J. Bhuta, B. Boehm, (2005) • A Method for Compatible COTS Component Selection, ICCBSS 10
  11. 11. Literature review • In recent years, researchers have focused on models and methods for reusable off-the-shelf (OTS) software selection. 1. Concentrate on evaluation and selection of specific software products. 2. Describe automated systems/tools that assist decision makers in various activities involved in software evaluation and selection. 3. Describe only criteria for software selection and methodology for software selection. 4. Relate to the evaluation of a single software attribute, quality or some quality subattribute, for a software product. 11
  12. 12. Software evaluation problem situations • Stamelos & Tsoukias (2003) • analyzed the contents of different ‘‘problem situations” and suggested a basic classification of software evaluation problem situations: Keep or Change Make or Buy Software certification Commercial product evaluation Software process evaluation. Tender evaluation Software system design selection. 12
  13. 13. Research method 1. 2. 3. 4. Inclusion criteria Search strategy and search Paper selection Data extraction 13
  14. 14. Inclusion criteria 1. Methodology for selecting software packages 2. Software evaluation criteria 3. Software evaluation technique 4. System/Tool to assist decision makers in evaluating software packages 14
  15. 15. Search strategy and search • Search Database ACM Portal Elsevier’s Science Direct IEEE Xplore SpringerVerlag 15
  16. 16. Paper selection 1. Initial selection from the search results, based on reading the abstract of the papers. 2. Final selection from the initially selected list of papers, based on reading of entire paper.  Reading all 130 papers in detail and considered 60 papers to be included in the final list for review. 16
  17. 17. Data extraction 17
  18. 18. Research Results • The contribution of the reviewed literature in the field of evaluation and selection. • Contribution of the literature in the field of evaluation and selection of the software packages (RQ1) • The Software packages is presented stage-based methodologies for selecting software packages. • Software selection methodologies (RQ2) 18
  19. 19. Contribution of the literature in the field of evaluation and selection of the software packages (RQ1) 19
  20. 20. Software selection methodologies (RQ2) Step 1: Step 2: Step 3: Step 4: • Determining the need for purchasing by vendor. • Short listing of candidate packages. • Eliminating most candidate package. • Using an evaluation technique of them. 20
  21. 21. Software selection methodologies (RQ2) Step 5: • Doing further scrutiny by obtaining trial copy. • Negotiating a contract specifying to terminate any agreement. Step 6: • Purchasing and implementing most appropriate software package. Step 7: 21
  22. 22. Research Results • Describes systems/tools to assist decision makers in evaluating software packages. • Systems/tools for evaluation and selection of software packages(RQ3) 22
  23. 23. Systems/Tools for evaluation and selection of software packages (RQ3) 23
  24. 24. Research Results • Techniques for evaluating software packages and software evaluation criteria are described in : • Software evaluation techniques (RQ4) • Evaluation criteria (RQ5) 24
  25. 25. Software evaluation techniques (RQ4) Analytic Hierarchy Process(AHP) Feature Analysis Weighted Average Sum (WAS) Fuzzy Based Approach 25
  26. 26. Analytic Hierarchy Process(AHP) Strengths: Weaknesses: • AHP enables decision makers to • AHP is time consuming because of the structure a decision making problem into a hierarchy, helping them to understand and simplify the problem. • It is flexible and powerful tool for handling both qualitative and quantitative multi-criteria problems. • AHP procedures are applicable to individual and group decision making. mathematical calculations and number of pair-wise comparisons that increases as the number of alternatives and criteria increases. • The decision makers need to re-evaluate alternatives when the number of criteria or alternatives are changed. • Ranking of alternatives depends on the alternatives consider edfor evaluation hence adding or deleting alternatives can lead to changes 26 in the final rank.
  27. 27. Feature Analysis Strengths: Weaknesses: • Evaluation can be done to any • Producing the single number from the required level of detail by organizing evaluation in different ways such as screening mode, case study, formal experiment and survey. individual scores may be misleading because many different combinations of numbers can produce the same aggregate score. • It is used not only for technical evaluation but also for evaluation of viability of supplier. 27
  28. 28. Weighted Average Sum (WAS) Strengths: • Main advantage of WAS is its ease of use. Weaknesses: • Weights to the attribute are assigned arbitrary and it is very difficult to assign weight when number of criteria is high. • To obtain a score using this method a common numerical scaling is required. • Difficulties emerge when WAS is applied to multi-dimensional MCDM problems. 28
  29. 29. Fuzzy Based Approach Strengths: • The decision makers can use linguistic terms to evaluate alternatives easily and intuitively. • It improves decision making procedure by accommodating the vagueness and ambiguity occurred during human decision making. Weaknesses: • Difficult to compute fuzzy appropriateness index values and ranking values for all alternatives. 29
  30. 30. Evaluation criteria (RQ5) Software quality characteristics 30
  31. 31. Evaluation criteria (RQ5) Vendor & Functional characteristics 31
  32. 32. Evaluation criteria (RQ5) Cost and Benefits & Hardware and Software 32
  33. 33. Evaluation criteria (RQ5) Opinions from technical and non-technical sources & Output 33
  34. 34. Limitations • This study has the usual limitations associated with any systematic literature review. • With respect to the search process, we have limited ourselves to English language studies and to four major electronic databases and search terms related to the terms ‘‘evaluation” and ‘‘selection”. • This strategy will not find non-English language papers, paper in many national journals and conferences, or papers that use unusual terminology. 34
  35. 35. Conclusions • This study provides an overview of the literature associated with evaluation and selection of software packages. • On the basis of literature review we propose generic software selection methodology and evaluation criteria. 35
  36. 36. Thanks for Your Listening Q&A 36

×