This document discusses quantitative and qualitative approaches to project evaluation and selection. It provides examples of quantitative methods like net present value comparisons, return on investment comparisons, and payback period comparisons. It also discusses limitations of quantitative methods for complex projects. Qualitative approaches like collective multifunctional evaluations that involve experts from different functions are also covered. The document recommends effective practices for project evaluation like defining success criteria, selecting the right evaluators, and taking an iterative approach.

1. Thamhain, H. J. “Project Evaluation and Selection”
The Engineering Handbook.
Ed. Richard C. Dorf
Boca Raton: CRC Press LLC, 2000
© 1998 by CRC PRESS LLC

2. 189
Project Evaluation and Selection
189.1 Quantitative Approaches to Project Evaluation and Selection
Net Present Value (NPV) Comparison • Return-on-Investment Comparison • Pay-Back Period (PBP)
Comparison • Pacifico and Sobelman Project Ratings • Limitations of Quantitative Methods
189.2 Qualitative Approaches to Project Evaluation and Selection
Collective Multifunctional Evaluations
189.3 Recommendations for Effective Project Evaluation and Selection
A Final Note
Hans J. Thamhain
Bentley College
For most organizations, resources are limited. The ability to select and fund the best projects with
the highest probability of success is crucial to an organization's ability to survive and prosper in
today's highly competitive environment. Project selections, necessary in virtually every business
area, cover activities ranging from product developments to organizational improvements,
customer contracts, R&D activities, and bid proposals. Evaluation and selection methods support
two principal types of decisions:
1. Judging the chances of success for one proposed project
2. Choosing the best project among available alternatives
Although most decision processes evaluate projects in terms of cost, time, risks and benefits,
such as shown in Table 189.1, it is often extremely difficult, if not impossible, to define a
meaningful aggregate measure for ranking projects regarding business success, technical risks, or
profit. Managers can use traditional, purely rational selection processes toward "right,"
"successful," and "best" only for a limited number of business situations. Many of today's complex
project evaluations require the integration of both analytical and judgmental techniques to be
meaningful.
Table 189.1 Typical Criteria for Project Evaluation and Selection
The criteria relevant to the evaluation and selection of a particular project depend on the specific project type and
business situation such as project development, custom project, process development, industry and market.
Typically, evaluation procedures include the following criteria:
• Development cost
• Development time
• Technical complexity
© 1998 by CRC PRESS LLC

3. • Risk
• Return on investment
• Cost benefit
• Product life cycle
• Sales volume
• Market share
• Project business follow-on
• Organizational readiness and strength
• Consistency with business plan
• Resource availability
• Cash flow, revenue, and profit
• Impact on other business activities
Each criterion is based on a complex set of parameters and variables.
Although the literature offers a great variety of project selection procedures, each organization
has its own special methods. Approaches fall into one of three principal classes:
1. Primarily quantitative and rational approaches
2. Primarily qualitative and intuitive approaches
3. Mixed approaches, combining both quantitative and qualitative methods
189.1 Quantitative Approaches to Project Evaluation and
Selection
Quantitative approaches are often favored to support project evaluation and selections if the
decisions require economic justification. Supported by numeric measures for simple and effective
comparison, ranking, and selection, they help to establish quantifiable norms and standards and
lead to repeatable processes. However, the ultimate usefulness of these methods depends on the
assumption that the decision parameterssuch as cash flow, risks, and the underlying economic,
social, political, and market factorscan actually be quantified. Typically, these quantitative
techniques are effective decision support tools if meaningful estimates of capital expenditures and
future revenues can be obtained and converted into net present values for comparison. Table
189.2 shows the cash flow of four project options to be used for illustrating the quantitative
methods described in this chapter.
Table 189.2 Cash Flow of Four Project Options or Proposals*
End of Year Do-Nothing Project Option Project Option Project Option
Option P1 P2 P3 P4
0 0 −1000 −2000 −5000
1 0 200 1500 1000
2 0 200 1000 1500
3 0 200 800 2000
4 0 200 900 3000
5 0 200 1200 4000
© 1998 by CRC PRESS LLC

4. Net cash flow 0 0 +3400 +7 500
NPV|N = 5 0 −242 +2153 +3 192
NPV|N = ∞ 0 +1000 +9904 +28 030
ROI|N = 5 0 20% 54% 46%
CB = ROINPV jn = 5 0 67% 108% 164%
NPBP ji = 0 0 5 1.5 3.3
NNPV ji 0 7.3 5 3.8
*Assuming an MARR of i = 10%
Note: The first line of negative numbers represents the initial investment at the beginning of the life
cycle.
Net Present Value (NPV) Comparison
This method uses discounted cash flow as the basis for comparing the relative merit of alternative
project opportunities. It assumes that all investment costs and revenues are known and that
economic analysis is a valid singular basis for project selection.
We can determine the net present value (NPV) of a single revenue, stream of revenues, and/or
costs expected in the future.
Present worth of a single revenue or cost (often called annuity, A) occurring at the end of period
n and subject to an effective interest rate i (sometimes referred to as discount rate or minimum
attractive rate of return, MARR) can be calculated as:
1
P W (A j i; n) = A = P Wn
(1 + i)n
Net present value of a series of revenues or costs, An, over N periods of time is as follows:
N
X X N
1
NPV(An j i; N ) = An = P Wn
n=1
(1 + i)n n=1
Table 189.2 applies these formulas to four project alternatives, showing the most favorable 5-year
net present value of $3192 for project option P4. (There are three special cases of net present
value: (1) for a uniform series of revenues or costs over N periods, NPV(Anj i, N) = A[(1+i)N -
1]/i(1+i)N; (2) for an annuity or interest rate i approaching zero, NPV = A £N; and (3) for the
revenue or cost series to continue forever, NPV = A/i.)
Return-on-Investment Comparison
Perhaps one of the most popular measures for project evaluation is the return on investment (ROI):
Revenue (R) ¡ Cost (C)
ROI =
Investment (I)
© 1998 by CRC PRESS LLC

5. It calculates the ratio of net revenue over investment. In its simplest form it entails the revenue on a
year-by-year basis relative to the initial investment (for example, project option 2 in Table 189.2
would produce a 20% ROI). Although this is a popular measure, it does not permit a comparative
evaluation of alternative projects with fluctuating costs and revenues. In a more sophisticated way
we can calculate the average ROI per year
" N #
X An
ROI(An ; In j N ) = =N
In
n=1
and compare it to the minimum attractive rate of return. All three project options, P2, P3 and P4,
compare favorably to the MARR of 10%, with project P3 yielding the highest average return on
investment of 54%. Or we can calculate the net present value of the total ROI over the project
lifecycle, also known as cost-benefit (CB). This is an effective measure of comparison, especially
for fluctuating cash flows. (Table 189.2 shows project 3 with the highest 5-year ROINPV of
108%.)
" N # " N #
X . X
ROINPV (An ; In j i; N ) = NPV(An j i; N ) NPV(In j i; N )
n=1 n=1
Pay-Back Period (PBP) Comparison
Another popular figure of merit for comparing project alternatives is the payback period (PBP). It
indicates the time period of net revenues required to return the capital investment made on the
project. For simplicity, undiscounted cash flows are often used to calculate a quick figure for
comparison, which is quite meaningful if we deal with an initial investment and a steady stream of
net revenue. However, for fluctuating revenue and/or cost streams, the net present value must be
calculated for each period individually and cumulatively added up to the "break-even point" in
time, NPBP, when the net present value of revenue equals the investment.
N
X N
X
NPV(An j i) ¸ NPV(In j i)
n=1 n=1
Pacifico and Sobelman Project Ratings
The previously discussed methods of evaluating projects rely heavily on the assumption that
technical and commercial success is ensured and all costs and revenues are predicable. Because of
these limitations, many companies have developed their own special procedures for comparing
project alternatives. Examples are the project rating factor (PR), developed by Carl Pacifico for
© 1998 by CRC PRESS LLC

6. assessing chemical products, and the project value factor (z), developed by Sidney Sobelman for
new product selections:
pT £ pC £ R
PR = z = (P £ TLC ) ¡ (C £ TD )
TC
Pacifico's formula is in essence an ROI calculation adjusted for risk. It includes probability of
technical success [.1 < pT < 1.0], probability of commercial success [.1 < pC < 1.0], total net
revenue over project life cycle [R], and total capital investment for product development,
manufacturing setup, marketing, and related overheads [TC]. The Sobelman formula is a modified
cost-benefit measure. It takes into account both the development time and the commercial life
cycle of the product. It includes average profit per year [P], estimated product life cycle [TLC],
average development cost per year [C], and years of development [TD].
Limitations of Quantitative Methods
Although quantitative methods of project evaluation have the benefit of producing relatively
quickly a measure of merit for simple comparison and ranking, they also have many limitations, as
summarized in Table 189.3.
Table 189.3 Comparison of Quantitative and Qualitative Approaches to Project Evaluation
Quantitative Methods Qualitative Methods
Benefits: Benefits:
Simple comparison, ranking, selection Search for meaningful evaluation metrics
Repeatable process Broad-based organizational involvement
Encourages data gathering and measurability Understanding of problems, benefits,
opportunities
Benchmarking opportunities Problem solving as part of selection process
Programmable Broad knowledge base
Input to sensitivity analysis and simulation Multiple solutions and alternatives
Multifunctional involvement leads to buy-in
Risk sharing
Limitations: Limitations:
Many success factors are nonquantifiable Complex, time-consuming process
Probabilities and weights change Biases via power and politics
True measures do not exist Difficult to proceduralize or repeat
Analysis and conclusions are often misleading Conflict- and energy-intensive
Methods mask unique problems and opportunities Do not fit conventional decision processes
Stifle innovative decision making Intuition and emotion dominates over facts
Lack people involvement, buy-in, commitment Justify wants over needs
Do not deal well with multifunctional issuesand Lead to more fact finding than decision making
dynamic situations
Pressure to act prematurely
© 1998 by CRC PRESS LLC

7. 189.2 Qualitative Approaches to Project Evaluation and
Selection
Especially for project evaluations involving complex sets of business criteria, the narrowly focused
quantitative methods must often be supplemented by broad-scanning; intuitive processes; and
collective, multifunctional decision making, such as Delphi, nominal group technology,
brainstorming, focus groups, sensitivity analysis, and benchmarking. Each of these techniques can
either be used by itself to determine the best, most successful, or most valuable option, or be
integrated into an analytical framework for collective multifunctional decision making, which is
discussed in the next section.
Collective Multifunctional Evaluations
Collective multifunctional evaluations rely on subject experts from various functional areas for
collectively defining and evaluating broad project success criteria, employing both quantitative
and qualitative methods. The first step is to define the specific organizational areas critical to
project success and to assign expert evaluators. For a typical product development project, these
organizations may include R&D, engineering, testing, manufacturing, marketing, product
assurance, and customer services. These function experts should be given the time necessary for
the evaluation. They also should have the commitment from senior management for full
organizational support. Ideally, these evaluators should have the responsibility for ultimate project
implementation, should the project be selected.
The next step is for the evaluation team to define the factors that appear critical to the ultimate
success of the projects under evaluation and arrange them into a concise list that includes both
quantitative and qualitative factors. A mutually acceptable scale must be worked out for scoring
the evaluation criteria. Studies of collective multifunctional assessment practices show that
simplicity of scales is crucial to a workable team solution. Three types of scale have produced most
favorable results in field studies: (1) 10-point scale, ranging from +5 = most favorable to -5 = most
unfavorable; (2) 3-point scale, +1 = favorable, 0 = neutral or can't judge, -1 = unfavorable; and (3)
5-point scale, A = highly favorable, B = favorable, C = marginally favorable, D = most likely
unfavorable, F = definitely unfavorable. Weighing of criteria is not recommended for most
applications, since it complicates and often distorts the collective evaluation.
Evaluators score first individually all of the factors that they feel qualified to make an expert
judgment on. Collective discussions follow. Initial discussions of project alternatives, their
markets, business opportunities, and technologies involved are usually beneficial but not necessary
for the first round of the evaluation process. The objective of this first round of expert judgments is
to get calibrated on the opportunities and challenges presented. Further, each evaluator has the
opportunity to recommend (1) actions needed for better assessment of project, (2) additional data
needed, and (3) suggestions that would enhance project success and the evaluation score. Before
meeting at the next group session, agreed-on action items and activities for improving the decision
process should be completed. With each iteration the function-expert meetings are enhanced with
more refined project data. Typically, between three and five iterations are required before a project
© 1998 by CRC PRESS LLC

8. selection can be finalized.
189.3 Recommendations for Effective Project Evaluation and
Selection
Effective evaluation and selection of project opportunities is critical to overall project success.
With increasing complexities and dynamics of the business environment, most situations are too
complex to use simple economic models as the sole basis for decision making. To be effective,
project evaluation procedures should include a broad spectrum of variables for defining the project
value to the organization.
Structure, discipline, and manageability can be designed into the selection process by grouping
the evaluation variables into four categories: (1) consistency and strength of the project with the
business mission, strategy, and plan; (2) multifunctional ability to produce the project results,
including technical, cost, and time factors; (3) success in the customer environment; and (4)
economics, including profitability. Modern phase management and stage-gate processes provide
managers with the tools for organizing and conducting project evaluations effectively. Table
189.4 summarizes suggestions that may help managers to effectively evaluate projects for
successful implementation.
Table 189.4 Suggestions for Effective Project Evaluation and Selection
1. Seek out relevant information. Meaningful project evaluations require relevant quality
information. The four categories of variables can provide a framework for establishing the
proper metrics and detailed data gathering.
2. Take top-down look; detail comes later. Detail is less important than information
relevancy and evaluator expertise. Don't get hung up on lack of data during the early phases
of the project evaluation. Evaluation processes should iterate. It does not make sense to
spend a lot of time and resources gathering perfect data to justify a "no-go"
decision.
3. Select and match the right people. Whether the project evaluation consists of a simple
economic analysis or a complex multifunctional assessment, competent people from those
functions critical to the overall success of the project(s) should be
involved.
4. Success criteria must be defined. Deciding on a single project or choosing among
alternatives, evaluation criteria must be defined. They can be quantitative, such as ROI, or
qualitative, such as the probability of winning a contract. In either case, these evaluation
criteria should cover the true spectrum of factors affecting success and failure of the
project(s). Only functional experts, discussed in point 3, are qualified to identify these
success criteria. Often, people from outside the company, such as vendors, subcontractors, or
customers, must be included in this expert group.
5. Strictly quantitative criteria can be misleading. Be aware of evaluation procedures based
only on quantitative criteria (ROI, cost, market share, MARR, etc). The input data used to
calculate these criteria are likely based on rough estimates and are often unreliable.
© 1998 by CRC PRESS LLC

9. Evaluations based on predominately quantitative criteria should at least be augmented with
some expert judgment as a "sanity check."
6. Condense criteria list. Combine evaluation criteria, especially among the judgmental
categories, to keep the list manageable. As a goal, try to stay within 12
criteria.
7. Communicate. Facilitate communications among evaluators and functional support groups.
Define the process for organizing the team and conducting the evaluation and selection
process.
8. Ensure cross-functional cooperation. People on the evaluation team must share a strategic
vision across organizational lines. They also must sense the desire of their host organizations
to support the project if selected for implantation. The purpose, goals, and objectives of the
project should be clear, along with the relationship to the business
mission.
9. Don't lose the big picture. As discussions go into detail during the evaluation, the team
should maintain a broad perspective. Two global judgment factors can help to focus on the
big picture of project success: (1) overall benefit-to-cost perception and (2) overall
risk-of-failure perception. These factors can be recorded on a ten-point scale: ¡5 to +5. This
also leads to an effective two-dimensional graphic display of competing project
proposals.
10. Do your homework between iterations. As project evaluations are most likely conducted
progressively, action items for more information, clarification, and further analysis surface.
These action items should be properly assigned and followed up, thereby enhancing the
quality of the evaluation with each consecutive iteration.
11. Stimulate innovation. Senior management should foster an innovative ambience for the
evaluation team. Evaluating complex project situations for potential success or failure
involves intricate sets of variables, linked among organization, technology, and business
environment. It also involves dealing with risks and uncertainty. Innovative approaches are
required to evaluate the true potential of success for these projects. Risk sharing by senior
management, recognition, visibility, and a favorable image in terms of high priority,
interesting work, and importance of the project to the organization have been found strong
drivers toward attracting and holding quality people to the evaluation team and gaining their
active and innovative participation in the process.
12. Manage and lead. The evaluation team should be chaired by someone who has the trust,
respect, and leadership credibility with the team members. Further, management can
positively influence the work environment and the process by providing some procedural
guidelines, charters, visibility, resources, and active support to the project evaluation
team.
A Final Note
Effective project evaluation and selection requires a broad-scanning process that can deal with the
risks, uncertainties, ambiguities, and imperfections of data available at the beginning of a project
cycle. It also requires managerial leadership and skills in planning, organizing, and
© 1998 by CRC PRESS LLC

10. communicating. Above all, evaluation team leaders must be social architects in unifying the
multifunctional process and its people. They must share risks and foster an environment that is
professionally stimulating and strongly linked with the support organizations eventually needed for
project implementation. This is an environment that is conducive to cross-functional
communication, cooperation, and integration of the intricate variables needed for effective project
evaluation and selection.
Defining Terms
Cross-functional: Actions that span organizational boundaries.
Minimum attractive rate of return (MARR): The annual net revenue produced on average by
projects in an organization as a percentage of their investments. Sometimes MARR is
calculated as company earnings over assets.
Net worth: Discounted present value of a future revenue or cost.
Phase management: Projects are broken into natural implementation phases, such as
development, production, and marketing, as a basis for project planning, integration, and
control. Phase management also provides the framework for concurrent engineering and
stage-gate processes.
Project success: A comprehensive measure, defined in both quantitative and qualitative terms,
that includes economic, market, and strategic objectives.
Stage-gate process: Framework for executing projects within predefined stages (see also phase
management) with measurable deliverables (gate) at the end of each stage. The gates
provide the review metrics for ensuring successful transition and integration of the project
into the next stage.
Weighing of criteria: A multiplier associated with specific evaluation criteria.
References
Brenner, M. 1994. Practical R&D project prioritization. Res. Technol. Manage. 37(5):38−42.
Bulick, W. J. 1993. Project evaluation procedures. Cost Eng. 35(10):27−32.
Menke, M. M. 1994. Improving R&D decisions and execution. Res. Technol. Manage.
37(5):25−32.
Obradovitch, M. M. and Stephanou, S. E. 1990. Project Management: Risk and
Productivity. Daniel Spencer, Bend, OR.
Remer, D. S., Stokdyk, S. B., and Van Driel, M. 1993. Survey of project evaluation techniques
currently used in industry. Int. J. Prod. Econ. 32(1):103−115.
Schmidt, R. L. 1993. A model for R&D project selection. IEEE Trans. EM. 40(4):403−410.
Shtub, A., Bard, J. F., and Globerson, S. 1994. Project Management: Engineering, Technology,
and Implementation. Prentice Hall, Englewood Cliffs, NJ.
Skelton, M. T. and Thamhain, H. J. 1993. Concurrent project management: a tool for technology
transfer. Proj. Manage. J. 26(4):41−48.
Ward, T. J. 1994. Which product is BE$T? Chemical Eng. 101(1):102−107.
© 1998 by CRC PRESS LLC

11. Further Information
The following journals are good sources of further information: Engineering Management
Journal (ASEM), Engineering Management Review (IEEE), Industrial Management (IIE), Journal
of Engineering and Technology Management, Project Management Journal (PMI), and
Transactions on Engineering Management (IEEE).
The following professional societies present annual conferences and specialty publications that
include discussions on project evaluation and selection: American Society for Engineering
Management (ASEM), Rolla, MO 65401, (314) 341-2101; Institute of Electrical and Electronic
Engineers (IEEE), East 47 St., New York, NY 10017-2394; and Project Management Institute
(PMI), Upper Darby, PA 19082, (610)734-3330.
© 1998 by CRC PRESS LLC