The document discusses evaluating the success of IT projects. It describes how success is defined differently by various stakeholders and outlines typical perspectives. It also discusses using scorecards and metrics to measure success factors like quality, cost, time, value and usefulness. User adoption is highlighted as a key measure of success. The document provides an example evaluation model and explains how balanced scorecards can be used to gauge user adoption and overall project success based on financial, customer, internal and learning/growth perspectives.

1. Successful Management of IT Projects 1/6
CKIT 554 Successful Management of IT Projects
Week 8: Evaluation models for IS/IT project success and user
adoption
Week 8 introduction
In this final week of the module, you will explore the methods
used to define and evaluate
success in IS/IT projects. In particular, you will examine the
scorecard method of estimating the
success of an IS/IT project, as well as early user adoption as a
measure of success.
Measuring success
At the beginning of a project, a project manager’s task is to
determine what is needed to deliver
a successful project. At the end, the task is to determine
whether those goals were achieved
(Lientz, 2011). However simple it may sound, this kind of
evaluation is not always a
straightforward task.
To evaluate any project, particularly an IS/IT project, two
questions must be answered: What is
the definition of success for this specific project? How do you
measure the success for this
specific project? A typical IS/IT project has a plethora of
stakeholders, each with a different

2. perception of success. In order to answer these two questions,
each group of stakeholders must
first answer this critical question: How can all stakeholders
work together toward success?
Table 1 illustrates typical perspectives of success in different
groups of stakeholders in an IS/IT
project.
Stakeholder groups Perspectives of success
Project team Clear project objectives, good
communication, monitoring and feedback;
adherence to budget, time and scope
requirements; project functionality,
project efficiency and team competence.
Project sponsors and clients Project deliverables on time,
budget as
planned and quality as desired;
acceptance, endorsement and satisfaction
of project.
Senior management and shareholders Good business and
financial sense;
technical competence of the team adding
to overall competence of the organisation;
new products/processes for market;
readiness for the future; organisation
mission and reputation enhancement.
Other stakeholders: customers, local and
national governments, local communities
User involvement; utility; safety; impact
on customer; customer service; increased
value to community or government.

3. Table 1 Stakeholders’ success perspectives
Evaluating IS/IT project success requires the use of systematic
frameworks to judge the
outcomes of a project. Many project managers create scorecards
to identify indicators of
Successful Management of IT Projects 2/6
success for their projects and create metrics that show the areas
of success and failure in a
project. User adoption is undoubtedly a key measure of success
in an IS/IT project.
Process-based measurements of IS/IT project success
Various models have emerged for the evaluation of IS/IT
projects. Each of these models is
appropriate for different project deliverables, types, sectors and
end users. IS/IT project
managers often use systematic frameworks to present various
alternative models and identify
the most important factors that will determine the success of a
project.
Process-based measurements typically address a number of
constraints aimed at meeting
specific project requirements, such as time, cost and quality. At
different phases of each project,
emphasis may be placed on one constraint over the others in
order to meet the overall project
deliverables. For example, during the development stage,
project managers judge ‘on-time’

4. delivery as the critical factor of success (to maintain the overall
schedule), while during testing
phases, ‘on-quality’ is the critical factor (to maintain the overall
product quality). IS/IT project
managers need to measure and manage this varied and staged
emphasis on project elements to
ensure the project’s overall success. A simple evaluation model
as shown in the Figure 2 can be
used to map the process-based factors of quality, cost and time
attributes for an IS/IT project to
the outcome-based factors of value, success and usefulness,
which stakeholders can more
easily understand.
Quality Strategic value
Maps to
Cost Time Success Usefulness
Process-based Outcome-based
Figure 1 Evaluation model for IS/IT projects
Importance of user adoption as early enabler to successful IS/IT
projects
IT/IS development project teams typically include users. Users
should represent employees from all
the functional areas and levels of an organisation who will
interact directly or indirectly with the final
system. Users must be from a wide variety of areas (not just
from one specific group or a single
department) to ensure interoperability and wide acceptance.
Project teams should also include
novice and experienced users, as many of the fundamental

5. issues identified in IS/IT projects often
involve basic intuitive functionality. Steve Jobs used such
approaches in the development of the
iPod, and he insisted on intuitive and simple functionality to
ensure user adoption (Arthur, 2012).
According to Schwalbe (2010), the lack of user adoption ‘leads
to problems managing scope creep
Successful Management of IT Projects 3/6
and controlling change’. Schwalbe suggests several steps to
deliver successful IS/IT projects,
including:
• Have a sponsor from the user organisation
• Make project information easily accessible and visible to the
project group and users
• Have users on the project team
• Collocate users with developers
• Deliver something to users and developers on regular basis
Successful early user adoption is also important in driving
continuous improvements and future
systems upgrade programmes, which ultimately ensure
longevity of the IS/IT system. Successful
user adoption also drives internal and external process
improvements which, when coupled with
systems optimisation initiatives, create gains in efficiency and
facilitate benchmarking.
It is prudent for project managers to adopt techniques such as
prototyping and use case modelling as
methods of understanding the deployment environment and its

6. unique requirements. These methods
ensure enough emphasis is placed on usability and user adoption
from the early stages of
development. Prototyping and use case modelling align the
IS/IT development process with user
expectations.
Using scorecards for determining IS/IT project success and
adoption
Scorecards, or balanced scorecards, are a simple and effective
way of gauging the level of user
adoption and success of the IS/IT project. Project managers use
scorecards to examine several
indicators of success and present the results of the analysis as
metrics. Scorecards are
particularly useful in gauging user adoption through stakeholder
involvement.
Defining and evaluating project success includes both
qualitative and quantitative indicators.
These indicators need to be appropriate for different types of
IS/IT projects to ensure that they
are correctly comparing and evaluating relevant elements. In
addition, they should be unbiased
(and not favour a positive outcome for the project team).
Successful Management of IT Projects 4/6
Balanced scorecards focus on the use of the most appropriate
information available during the project
and measure both financial and non-financial indicators in the
following key areas: financial, customer,
internal business processes and learning and growth (Kaplan

7. and Norton, 1996). Figure 2 shows the four
organisational perspectives.
Figure 2 The four perspectives of balanced scorecard (Kaplan
and Norton, 1996)
Project managers use balanced scorecards to gauge whether or
not the project was successful
by looking at each of the four key areas. For a specific project,
the relevant items from the four
key areas can be displayed in a cause-and-effect chain. For
example, the map in Figure 3
(Kaplan & Norton, 2000) shows the picture of drivers that
enable an organization to visually
illustrate its objectives, initiatives and targets—the measures
used to assess the IS/IT project’s
performance.
Financial Perspective
Are we meeting the expectations of
our shareholders?
Customer Perspective
Are we satisfying our customers?
Internal Process Perspective
Are we doing things right and
doing the right things?
Learning and Growth

8. Perspective
Are we sustaining our ability to
change and improve for the future?
Organisational
Vision and
Strategy
Successful Management of IT Projects 5/6
Financial perspective
Customer perspective
Internal
perspective
Learning and growth
perspective
Figure 3 IS/IT project strategy map (Kaplan and Norton, 2000)
In conclusion, project managers use a variety of methods to
define and evaluate success in IS/IT
projects. Many project managers find balanced scorecards and
early user adoption particularly
Innovate (Create
new processes and

9. products)
Improve (Improve
and leverage
quality)
Implement (Develop
solutions)
Improve
client/customer
satisfaction
Employee
competencies
Develop a climate
of action
Put technology to
work
Improve
client/customer
value

10. Maximise
business value
value
Project operational
excellence
IT project team
leadership
Value from new
processes and
products
Improve IT asset
utilisation across
the organisation
Successful Management of IT Projects 6/6
accurate tools of measuring success in IS/IT projects.
References
Arthur, C. (2012) Digital wars: Apple, Google, Microsoft and

11. the battle for the Internet. London:
Kogan Press
Kaplan, R. & Norton, D. (2000) ‘Having trouble with your
strategy? Then map it’, Harvard
Business Review, September–October, pp. 1-12
Kaplan, R. & Norton, D. (1996) ‘Using the balanced scorecard
as a strategic management
system’, Harvard Business Review, January–February, pp. 75-
85
Lientz, B.P. (2011) Information technology project
management. New York: Palgrave Macmillan
Schwalbe, K. (2010) Managing information technology projects.
6th ed. Boston: Cengage
Learning
Scanned by CamScanner
Scanned by CamScanner
( a s e s a n d e x a m p
l e s 7t :
A $ M D Ba n k
x m a n a ge m e n t B5 e the pr o je c t t e a m s fo r b o t h
t h e l e g a c y a n d n e w s y s t e m s s t a r t e db a i Se p
a r a t e pr o je c t t e a m s w e r e e s t a b l i s h e d f o r e

12. a c h N o r o l e s
,
r e s p o n s ib i li t i e s
, Or
e p e r ie n c e a n d k n e w w h a t t o d o
- w em e s Aa bl is he d T h e p r o je c t le a d e r a s s u m e
d t h a t e v e r yo n e h a d p r e v i o u s pr o je c t
Th e - b e ga n t o d e t e r i o r a t e T h e n e w s y t e m p r
o je c t t e a m h a d n e v e r w o r ke d- - M a n y w e r e j u
n i o r p e o p l e w h o l a c k e d p r o je c t m a n a g e m e
n t e x p e r i e n c eT h e , ( w a s n o t ge t r in g o r ga n i z
e d T h e t e a m j u s t s t a r t e d i n a n d b e g a n o n t a
s ks T he.e p cy s t e m t e a m w a s t h e r e v e r s e Tie a m
m e m b e r s h a d o p e r a t e d i n a b u r e a u c r a t ic
- a n d m a io r i n t e r f a c e w o r k T h e y t r e a t e d t h e
n e w w o r k l ik e t h e o l d T h e ir p r o je c t
- w e r e a lm o s t x ) c ia l O c c a s io n s
t - t o tha t de s c r i be d a b o v e o n t h e n e w t e a m M
e m b e r s o f t h is t e a m
w a e t lhe l e a s t r e s is t a n t a n d m o s t a c c e p t in g
o f o r ga n i z a t io n O n c e t h is p r o c e s s w a s
Scanned by CamScanner
Discussion 1: Evaluation at Astro Bank (Focused Discussion)
Top of Form

13. IS/IT project managers use evaluation processes to determine
the success of a project. This kind of evaluation is not always a
straightforward task. In this Discussion, you will determine
evaluation criteria in an example project from a case study.
To complete this Discussion:
1. Refer to the Astro Bank case study on p.138, Part two:
Project planning as well as discussion under section ‘How do
you keep team members?’ on page 137 of the Lientz
text, Information Technology Project Management. Identify and
critically analyse the criteria the company should use in
evaluating the success of the integrated system (formed by the
new IS/IT system and the legacy system). (400 words/ 3
references)
2. Three (3) Point you would put forward in a Focus debate
relating to this topic giving prior experience /views extend the
discussion into new but relevant areas also. ( Short answer no
words limit required/ 1 reference required)
Bottom of Form
Discussion 2: The balanced scorecard (Focused Discussion)
Top of Form
Project managers can use one of several approaches to
measuring the quality and success of an IS/IT project. For
example, balanced scorecards are a simple and effective way of
gauging the level of user adoption and success of an IS/IT
project. In this Discussion, you will explore how using a
balanced scorecard could improve the project performance of an
example project in a case study.
To complete this Discussion:
1. The Week 8 Lecture Notes draw attention to the four
perspectives of the balanced scorecard that can translate an
organisation’s vision and strategy (as shown in Figure 2 of the
Lecture Notes). Explain how you could apply them to improve

14. the IS/IT project performance of the Astro Bank case study on
p.138, Part two: Project planning, considering also
discussion under section ‘How do you keep team members?’ of
the Lientz text, Information Technology Project Management.
(400 words /3 references )
2. Three (3) Point you would put forward in a Focus debate
relating to this topic extend the discussion into new but relevant
areas also. ( Short answer no words limit required/ 1 reference
required)
Bottom of Form