Project Audit/Evaluation/Review
• A more or less formal inquiry into any aspect of the
project
• A major vehicle for evaluation
• “Evaluation” – to set the value of; to appraise
• “Project evaluation” – appraises the progress and
performance of a project compared to that project’s
planned progress and performance, or compared to
the progress and performance of other similar projects
12-3
The Project Audit
 What and why
 Benefits of a project audit
 Judging success and failure
 Determining project objectives
 Contents and format of a project audit
 Project Audit Life Cycle
 Responsibilities of an auditor
12-4
What is a Project Audit, & Why Is It Done?
 A formal inquiry into any or all aspects of a project
 Possible reasons:
 Revalidate the business feasibility of the project
 Reassure top management
 Confirm readiness to move to next phase of project
 Investigate specific problems
12-5
Some Specific Benefits of a Well-Done Project
Audit
 Identify problems earlier
 Clarify performance/cost/schedule relationships
 Improve project performance
 Identify future opportunities
 Evaluate performance of project team
 Reduce costs
 Inform client of project status/prospects
 Reconfirm feasibility of/commitment to project
12-6
Judging a Project’s Success
 To what extent is a project meeting its objectives?
 Efficiency: Does the project use resources in a cost-
effective manner? Cost efficiency? Schedule efficiency?
 Customer impact/satisfaction: Quality, timeliness,
customer satisfaction, meeting/exceeding specifications.
 Business success: Meeting expectations in ROI, market
share, cash flow
 Future potential: Will project lead to future business
prospects?
12-7
The Difference Between Project Success &
Failure
 Audits of 110 projects over 11 years reveal four basic
differences between success and failure
 Objectivity in design, scope, cost and schedule
 Experienced people throughout project
 Authority commensurate with responsibility
 Clear responsibility and accountability
12-8
Determining What the Project
Objectives Really Are
 Explicit objectives are easy to find
 Cost, schedule, performance specs
 Profit targets
 Ancillary objectives are not
 Examples include retaining employees, maintaining a
customer, getting a “foot in the door,” developing a new
capability, blocking a rival
12-9
Ancillary Objectives are Important, but Often
Obscure
 If an audit ignores ancillary objectives, it will draw
an incomplete picture
 But people tend to disguise ancillary objectives.
Why?
 If not explicit, how can it be judged a failure?
 People and teams may have their own goals and
priorities
 The stronger the project culture, the greater the
suspicion toward outsiders, e.g., auditors
12-10
Costs of Project Audits
 While audits offer benefits, they aren’t free
 Some costs are obvious, others less so
 Salaries of auditors and staff
 Distraction from project work
 Before and during the audit
 Anxiety and morale within the project
 Cost of outside experts
12-11
Timing of the Audit
 Early audits tend to focus on technical issues, and tend
to benefit the project
 Later audits lean toward cost and schedule, and tend
to benefit the parent organization
 Transfer of lessons learned to other projects
12-12
Contents of a Project Audit
 Format can vary, but six areas should be covered
 1. Project status, in all dimensions
 2. Future projections
 3. Status of crucial tasks
 4. Risk assessment
 5. Information relevant to other projects
 6. Limitations of the audit
12-13
A Format for a Project Audit
 Introduction
 Including project objectives
 Also audit assumptions, limitations
 Current project status
 Cost
 Schedule
 Progress/Earned Value
 Quality
12-14
Format for Project Audit (cont’d)
 Future Project Status
 Conclusions and recommendations
 Critical Management Issues
 A Pareto approach
 Risk Management
 Major threats to project success
 Appendices
12-15
The Project Audit Life-Cycle
 Like the project itself, the audit has a life cycle
 Six basic phases:
 1. Project audit initiation
 Focus and scope of audit; assess methodologies, team
members required
 2. Baseline Definition
 Determine the standards against which performance will be
measured
12-16
The Audit Life Cycle (cont’d)
 3. Establishment of Audit Database
 Gathering/organizing pertinent data
 Focus on what’s necessary
 4. Data Analysis
 The judgment phase
 Comparison of actuals to standard
12-17
The Audit Life Cycle (cont’d)
 5. Audit Report Preparation
 Present findings to PM first
 Then, prepare final report
 6. Audit Termination
 Review of audit process
 Disbanding of team
12-18
Responsibilities of a Project
Auditor
 As in medicine, “first do no harm”
 Be truthful, upfront with all parties
 Maintain objectivity and independence
 Acknowledge entering biases
 Project confidentiality
 Limit contacts to those approved by management
12-19
Baseline Marketing Data, Figure 12-2
Purposes of Evaluation
• Four dimensions of project success
– Efficiency in meeting both the budget and schedule
(meeting cost, time and scope objectives)
– Customer impact/satisfaction (loyalty, repurchase,
fulfilling customer’s needs, actual use by the customer)
– Business/direct success (level of commercial success,
market share; yields, cycle times, quality, processing
steps)
– Future potential (factors relating to opening a new
market, developing a new line of products and services,
developing a new technology, skills, competencies)
Purposes of Evaluation, 2
 To help translate the achievements of the project’s
goals into a contribution to the parent organization’s
goals.
 To do this, all facets of the project are studied in order to
identify and understand the project’s strengths and
weaknesses.
 It is the equivalent of an application of Six-Sigma or
TQM to project management.
 Result in a set of recommendations for the project’s
direct goals
Purposes of Evaluation, 3
 To review contributions of ancillary/ unplanned goals:
 Improve understanding of the ways in which projects may be of value to the
organization
 Improve the process for organizing and managing projects (project management
maturity)
 Provide information and experience for entering new markets
 Provide a congenial environment in which project team members can work creatively
together
 Identify organizational strengths and weaknesses in project-related personnel,
general management and decision-making techniques and systems
 Identify and improve the response to risk factors in the firm’s use of projects
 Allow access to project policy decision-making by external stakeholders
 Improve the way projects contribute to the professional growth of project team
members
 Identify project personnel who have high potential for managerial leadership
The Project Audit
 A thorough examination of the management of a
project, its methodology and procedures, its records,
its properties, its budgets and expenditures and its
degree of completion.
 Different from and broader than a financial audit (use
and conservation of resources) and technical audit
 It may deal with the project as a whole, or only with a
part of the project
The Formal Audit Report
 Current status of the project
 Does the work actually completed match the planned
level of completion?
 Future status
 Are significant schedule/scope changes likely? If so,
indicate the nature of the changes
 Status of crucial tasks
 What progress has been made on tasks that could
decide the success or failure of the project?
The Formal Audit Report, 2
 Risk assessment
 What is the potential for project failure or monetary
loss?
 Information pertinent to other projects
 What lessons learned from the project being audited
can be applied to other projects being undertaken by the
organization?
 Limitations of the audit
 What assumptions or limitations affect the data in the
audit?
The Project Audit Life Cycle
 Project Audit Initiation
 Project Baseline Definition
 Establishing an Audit Database
 Preliminary Analysis of the Project
 Audit Report Preparation
 Project Audit Termination
Responsibilities of the Project
Auditor/Examiner
 “Tell the truth”: objective, ethical manner
 Walker et al (1980): independence:
 Assemble a small team of experienced experts
 Familiarize the team with the requirements of the
project
 Audit the project on-site
 After completion, debrief the project’s management
 Produce a written report to the PM and project team for
their response
 Follow up to see if the recommendations have been
implemented

2 a. project auditing

  • 2.
    Project Audit/Evaluation/Review • Amore or less formal inquiry into any aspect of the project • A major vehicle for evaluation • “Evaluation” – to set the value of; to appraise • “Project evaluation” – appraises the progress and performance of a project compared to that project’s planned progress and performance, or compared to the progress and performance of other similar projects
  • 3.
    12-3 The Project Audit What and why  Benefits of a project audit  Judging success and failure  Determining project objectives  Contents and format of a project audit  Project Audit Life Cycle  Responsibilities of an auditor
  • 4.
    12-4 What is aProject Audit, & Why Is It Done?  A formal inquiry into any or all aspects of a project  Possible reasons:  Revalidate the business feasibility of the project  Reassure top management  Confirm readiness to move to next phase of project  Investigate specific problems
  • 5.
    12-5 Some Specific Benefitsof a Well-Done Project Audit  Identify problems earlier  Clarify performance/cost/schedule relationships  Improve project performance  Identify future opportunities  Evaluate performance of project team  Reduce costs  Inform client of project status/prospects  Reconfirm feasibility of/commitment to project
  • 6.
    12-6 Judging a Project’sSuccess  To what extent is a project meeting its objectives?  Efficiency: Does the project use resources in a cost- effective manner? Cost efficiency? Schedule efficiency?  Customer impact/satisfaction: Quality, timeliness, customer satisfaction, meeting/exceeding specifications.  Business success: Meeting expectations in ROI, market share, cash flow  Future potential: Will project lead to future business prospects?
  • 7.
    12-7 The Difference BetweenProject Success & Failure  Audits of 110 projects over 11 years reveal four basic differences between success and failure  Objectivity in design, scope, cost and schedule  Experienced people throughout project  Authority commensurate with responsibility  Clear responsibility and accountability
  • 8.
    12-8 Determining What theProject Objectives Really Are  Explicit objectives are easy to find  Cost, schedule, performance specs  Profit targets  Ancillary objectives are not  Examples include retaining employees, maintaining a customer, getting a “foot in the door,” developing a new capability, blocking a rival
  • 9.
    12-9 Ancillary Objectives areImportant, but Often Obscure  If an audit ignores ancillary objectives, it will draw an incomplete picture  But people tend to disguise ancillary objectives. Why?  If not explicit, how can it be judged a failure?  People and teams may have their own goals and priorities  The stronger the project culture, the greater the suspicion toward outsiders, e.g., auditors
  • 10.
    12-10 Costs of ProjectAudits  While audits offer benefits, they aren’t free  Some costs are obvious, others less so  Salaries of auditors and staff  Distraction from project work  Before and during the audit  Anxiety and morale within the project  Cost of outside experts
  • 11.
    12-11 Timing of theAudit  Early audits tend to focus on technical issues, and tend to benefit the project  Later audits lean toward cost and schedule, and tend to benefit the parent organization  Transfer of lessons learned to other projects
  • 12.
    12-12 Contents of aProject Audit  Format can vary, but six areas should be covered  1. Project status, in all dimensions  2. Future projections  3. Status of crucial tasks  4. Risk assessment  5. Information relevant to other projects  6. Limitations of the audit
  • 13.
    12-13 A Format fora Project Audit  Introduction  Including project objectives  Also audit assumptions, limitations  Current project status  Cost  Schedule  Progress/Earned Value  Quality
  • 14.
    12-14 Format for ProjectAudit (cont’d)  Future Project Status  Conclusions and recommendations  Critical Management Issues  A Pareto approach  Risk Management  Major threats to project success  Appendices
  • 15.
    12-15 The Project AuditLife-Cycle  Like the project itself, the audit has a life cycle  Six basic phases:  1. Project audit initiation  Focus and scope of audit; assess methodologies, team members required  2. Baseline Definition  Determine the standards against which performance will be measured
  • 16.
    12-16 The Audit LifeCycle (cont’d)  3. Establishment of Audit Database  Gathering/organizing pertinent data  Focus on what’s necessary  4. Data Analysis  The judgment phase  Comparison of actuals to standard
  • 17.
    12-17 The Audit LifeCycle (cont’d)  5. Audit Report Preparation  Present findings to PM first  Then, prepare final report  6. Audit Termination  Review of audit process  Disbanding of team
  • 18.
    12-18 Responsibilities of aProject Auditor  As in medicine, “first do no harm”  Be truthful, upfront with all parties  Maintain objectivity and independence  Acknowledge entering biases  Project confidentiality  Limit contacts to those approved by management
  • 19.
  • 20.
    Purposes of Evaluation •Four dimensions of project success – Efficiency in meeting both the budget and schedule (meeting cost, time and scope objectives) – Customer impact/satisfaction (loyalty, repurchase, fulfilling customer’s needs, actual use by the customer) – Business/direct success (level of commercial success, market share; yields, cycle times, quality, processing steps) – Future potential (factors relating to opening a new market, developing a new line of products and services, developing a new technology, skills, competencies)
  • 21.
    Purposes of Evaluation,2  To help translate the achievements of the project’s goals into a contribution to the parent organization’s goals.  To do this, all facets of the project are studied in order to identify and understand the project’s strengths and weaknesses.  It is the equivalent of an application of Six-Sigma or TQM to project management.  Result in a set of recommendations for the project’s direct goals
  • 22.
    Purposes of Evaluation,3  To review contributions of ancillary/ unplanned goals:  Improve understanding of the ways in which projects may be of value to the organization  Improve the process for organizing and managing projects (project management maturity)  Provide information and experience for entering new markets  Provide a congenial environment in which project team members can work creatively together  Identify organizational strengths and weaknesses in project-related personnel, general management and decision-making techniques and systems  Identify and improve the response to risk factors in the firm’s use of projects  Allow access to project policy decision-making by external stakeholders  Improve the way projects contribute to the professional growth of project team members  Identify project personnel who have high potential for managerial leadership
  • 23.
    The Project Audit A thorough examination of the management of a project, its methodology and procedures, its records, its properties, its budgets and expenditures and its degree of completion.  Different from and broader than a financial audit (use and conservation of resources) and technical audit  It may deal with the project as a whole, or only with a part of the project
  • 24.
    The Formal AuditReport  Current status of the project  Does the work actually completed match the planned level of completion?  Future status  Are significant schedule/scope changes likely? If so, indicate the nature of the changes  Status of crucial tasks  What progress has been made on tasks that could decide the success or failure of the project?
  • 25.
    The Formal AuditReport, 2  Risk assessment  What is the potential for project failure or monetary loss?  Information pertinent to other projects  What lessons learned from the project being audited can be applied to other projects being undertaken by the organization?  Limitations of the audit  What assumptions or limitations affect the data in the audit?
  • 26.
    The Project AuditLife Cycle  Project Audit Initiation  Project Baseline Definition  Establishing an Audit Database  Preliminary Analysis of the Project  Audit Report Preparation  Project Audit Termination
  • 27.
    Responsibilities of theProject Auditor/Examiner  “Tell the truth”: objective, ethical manner  Walker et al (1980): independence:  Assemble a small team of experienced experts  Familiarize the team with the requirements of the project  Audit the project on-site  After completion, debrief the project’s management  Produce a written report to the PM and project team for their response  Follow up to see if the recommendations have been implemented