The document discusses the emerging demand for audits of business projects from boards and senior management. It notes that boards are increasingly interested in improving performance and looking for guidance on IT projects, but often rely too heavily on management and external consultant recommendations without thorough review. Standards for IT project management exist but provide little practical guidance for ensuring projects deliver above-average returns. The document argues that a new standard focusing on evaluating expected benefits, sponsorship, accountability and realizing benefits could help meet the emerging demand for better governance and outcomes from business projects.

1. The Emerging Demand
for Business Project Audits
Presented by: Dr Raymond Young
DrRYoung@gmail.com
ISACA Technical Session
Sydney, 14 March 2007
© Raymond Young The Emerging Demand for Business Project Audits No 1

2. To ensure management
is focussed on above average returns
taking account of risk (Hilmer) AWB,
James Hardie
Backlash against risk
Corporate Sarbanes-Oxley
Governance CLERP9, ASX,
AS8000
ENRON,
True and Worldcom
Legislative fair view avoid box ticking
The Emerging Demand Turnbull (1999)
Hampel
Risks (1998)
for Controls
Directors are
responsible
for risk
Business Project Audits Testing
Bosch
(1995)
KING
Disasters & COCO
corporate
collapses
COSO
Cadbury
(1992)
© Raymond Young The Emerging Demand for Business Project Audits No 2

3. An emerging board level demand
• Boards appear to be genuinely interested in improving their
performance (Leblanc 2005)
– PRO:NED have identified IT Governance as an emerging board trend.
– Almost one third of UK firms include an executive with e-business
responsibilities on the board (CBI, 2001). It is more than 60% in US
companies
• Boards are looking for guidance with respect to IT (Young and Jordan 2002)
– In the next few years, E-commerce risks will be among the most
important that companies will face (SRBI, 2001)
– “there’s a lot of discussion by directors … [ecommerce] …the company
might be left behind” … “some board members were saying we should
be doing more, some less … nobody much was up the curve so the
board members were all trying to figure it out”.
– Most companies recognised a much higher level of risk because of the
uncertainties involved and relied on external advice to validate any
strategy proposed by management
– … the main considerations are “the fear of the consequences of making
a mistake, a fear of costing multiple times more than what anybody
planned, a fear they don’t have the expertise” and the lack of
successful models to guide decision making
© Raymond Young The Emerging Demand for Business Project Audits No 3

4. Who do directors trust for advice?
• Directors recognised a need for an understanding of technology but their
experience with IT people was disappointing.
“With technocrats, the only three things you can be sure of are: nothing would get
finished on time, it would always cost vastly more than predicted and it would
never do what it was promised to do”.
• Main sources of advice
“I tend to be relying more on outside experts than I would normally do when
assessing a project”
“there is a tendency to accept the recommendations of management, which tend to
echo the recommendations of the consultants …there is no thorough analytical
review in the way you would have in other areas where the directors know what’s
going on”
“the board members I talk to are very comfortable if there is someone taking
responsibility and particularly comfortable if it’s McKinsey or one of the major
firms”
• Recognised deficiencies in the advice
“are the boards [receiving] summaries, no … it's difficult for directors to make good
decisions” … “I don’t see anyone doing a consequences analysis documenting
what if we don't do it and what if we do it” … “management is gathering tons of
information and don't know how to present it”
“for all the Standards that do exist, there is no guidance on how to go about making
the decisions to implement an ecommerce strategy”
© Raymond Young The Emerging Demand for Business Project Audits No 4

5. IT & Project Standards
vs Business Project Standards
• Best Practice Standards “I have checked with ISACA and they
– PMBOK, PRINCE2 agree with your conclusion about
– BS15000, ITIL [lack of] supporting evidence.”
– ISO/IEC 12207, ISO/IEC
12207
– ISO17799, NIST handbook • AS8015
– COBIT – “makes it sound doable”
• Challenges – Under consideration for
– “Little practical utility” fast-track adoption by ISO
(Lyytinen 1987) • AS8016
– “no consistent impact on
success” (Kraemer and King 1986) – HB280–2006
– Overemphasise technical & – ValIT
project management
considerations (Lucas 1975, Sauer
1999, Young 2005)
– Untested against objective
criteria (Checkland 1981, Strassman
1995, Young & Jordan 2005)
© Raymond Young The Emerging Demand for Business Project Audits No 5

6. HB280: a refocus on above average returns
Project Management vs. Project Success (de Wit 1985)
• project management success is less important (Baccarini 1999)
• “benefits are not delivered or realised by the project manager and project team, they
require the actions of operations management.” (Cooke-Davis 2002)
• there is not a strong relationship between project management success and project
success or between project management failure and project failure (Markus et. al. 2000)
Scope of project success
Scope of project
management success
1 2 3 4 5 6
1 2 3 4 5 6
Initiation Planning Development Implementation Benefit Closedown
© Raymond Young The Emerging Demand for Business Project Audits No 6

7. HB280: Case Studies
What is the most important success factor?
Hypothesis T-Serv T-Media ABS Agency SkyHigh Conclusion
Top management
1.
support      Strong support
High level
2.
project planning      Inconclusive
Project
3.
methodologies   -   Inconclusive
User
4.
involvement      Inconclusive
5. Project staff      Not supported
© Raymond Young The Emerging Demand for Business Project Audits No 7

8. HB280: Inconsistent criteria for success
EDP Success: MIS Success: SIS Success:
Technical focus User focus Organisational focus
Information
Quality
Intention
Use
to use
System
Net Benefits
Quality
User Satisfaction
Service
Quality
Delone, W. H. and McLean, E. R. (2003)
The Delone and McLean Model of Information Systems Success: a ten-year update
Journal of Management Information Systems, 19:4, 9-30.
© Raymond Young The Emerging Demand for Business Project Audits No 8

9. Average Returns
(Young 2006b)
4–10x (Brynjolfsson & Hitt 1998)
or ROI
30% (Garrity 1963)
30%
Some OK
10-20% (Clegg et al 1997)
35%
2/3 of projects
30-40% No
(Willcocks and Margetts 1994)
deliver no benefits
whatsoever
Fail
15% (Standish 1999,2003)
© Raymond Young The Emerging Demand for Business Project Audits No 9

10. Above Average Returns
(Young 2006b)
ROI
ROI 240%
ROI 135%
30%
OK
OK
Some OK
Current Better Excellent
Performance Performance Performance
Some
Cancel
(68% under) (43% under) (15% cancelled)
Fail
No
National Implication
Fail
additional $10.5B-$21B pa (1.6 - 3.1%
GDP)
© Raymond Young The Emerging Demand for Business Project Audits No 10

11. HB280:2006
A better way to understand projects
2
Evaluate
1
Initiate Top
Management Direct &
Support
Monitor
3 4
6
ICT Projects 5 Changed
Operations ICT Operations
Changed
Business processes Business Processes
© Raymond Young The Emerging Demand for Business Project Audits No 11

12. HB280 are the
1. What
Recommendations
expected business 2. How much change
benefits? is required to realise
the benefits?
2
Evaluate
1
Initiate Top
Management Direct &
3. Who should Support
Monitor
sponsor the 3 4
project?
6. Are the benefits
realised?
4. Who will be
6
accountable for the
ICT
benefits and how will Projects 5 Changed
Operations ICT Operations
they be measured and
rewarded? 5. Is the culture right for Changed
ie how will realisation of benefits
Business processes unexpected issues to beProcesses
Business
be measured? raised?
© Raymond Young The Emerging Demand for Business Project Audits No 12

13. What is Project Governance?
Hard & Soft prescriptions
Business Plan
Champion Project Team
Project Plan •PM, consultants, IT,
•Operations
3.Support 4.Support
1.Initiate 2.Evaluate 5.Implement 6.Benefits
Meta-structural structural
Culture
Sponsor
Project-image Governance Structure
Motivation Beliefs
•Consensus
•Passion •Reward Accountability
•Will to change
© Raymond Young The Emerging Demand for Business Project Audits No 13

14. Conclusion
possible indicators of the emerging demand for
business project audits
• Appointment of board members with IT experience
• McKinsey or one of the major firms
– offer project audit services
– UKIIA
– c.f. Jed Simms at Capability
• Successful Models to guide decision making
– AS8015, AS8016, ISO ?
– ARC linkage grant, five collaboration partners contributing $550,000 over 3 years
($100,000 in cash)
• When ‘Business projects’ enters the business jargon
– ValIT vs PMBOK
– When project managers realise they should not be king.
– When business managers finally step up to the plate
• Major negative press
– an ‘James Hardie’ relating to IT c.f. API
– Public sector exposure?
© Raymond Young The Emerging Demand for Business Project Audits No 14

15. Key References
• Cooke-Davies, T. (2002). "The "real" success factors on projects." International Journal of Project
Management 20: 185-190.
• de Wit (1985). "Measurement of project success." International Journal of Project Management
6(3): 164-170.
• Leblanc, R. and J. Gillies (2005). Inside the Boardroom: the coming revolution in corporate
governance. Toronto, John Wiley and Sons.
• Lucas, H. C. (1975). Why Information Systems Fail. New York, Columbia University Press.
• Markus, M. L., S. Axline, et al. (2000). "Learning from adopters' experience with ERP: problems
encountered and success achieved." Journal of Information Technology 15: 245-265.
• SRBI (2001). The E-Frontier: New Challenges to Corporate Risk Management. New York, SRBI
Study for The St. Paul Companies.
• Strassmann, P. A. (1995). The Politics of Information Management. New Canaan, CT.,
Information Economics Press.
• Young, R. (2006a). HB 280-2006 Case Studies - How Boards and Senior Management Have
Governed ICT Projects to Succeed (or Fail). Sydney, Standards Australia.
• Young, R. (2006b). What is the ROI for IT Project Governance? Engaging the board and top
management. 2006 IT Governance International Conference, Auckland, New Zealand.
• Young, R. and E. Jordan (2005). The implications of Australian ICT Governance Standards for
COBIT. 2005 IT Governance International Conference, Auckland, New Zealand.
• Young, R. C. and E. Jordan (2002). Lifting the Game: Board views on e-commerce risk. IFIP
TG8.6 the adoption and diffusion of IT in an environment of critical change, Sydney, Pearson
Publishing Service.
© Raymond Young The Emerging Demand for Business Project Audits No 15

16. Questions
Please provide your business card for
a copy of this presentation
Warning: You will be “researched”
© Raymond Young The Emerging Demand for Business Project Audits No 16