The document discusses value-for-money auditing and assessing the value of IT investments and operations. It defines value-for-money as operating economically, efficiently, and effectively. It provides definitions and examples of economy, efficiency, and effectiveness. The presentation discusses how IT value is linked to risk, governance, and business objectives. Tools like Val IT, Risk IT, and COBIT are presented as frameworks to assess IT value, risk, and governance. The goal is to ensure IT supports business goals and priorities rather than existing for its own sake.

1. Value-For-Money IT
Are You Operating Economically,
Efficiently and Effectively?
Presenters:
Ron Foster, CISA, CIA, PMP, CMA
Auditor General – City of Oshawa
Paul Wallis, CMA, CIA, CISA
Director, Internal Audit – Region of Peel

2. Agenda
 Value for Money Auditing - What is it?
 IT Value - What is it? - Why is it Important? How
does it Link to Risk and Governance?
 Value IT Example
 Value for Money Audit Process Models and
Tools

3. Value-for-Money Auditing
Lets Clear The Air!
(It is More than Public
Sector Auditing)
Definitions??

4. Classic Definition
 Value-for-Money Auditing is one of the Three
Elements of Comprehensive Auditing.
 Comprehensive Auditing Embraces Three
Related but Separate Aspects of Public Sector
Accountability Including:
 Financial Reporting
 Compliance with Authorities
 The Economical, Efficient and Effective
Management of Public Funds and Resources

5. Brief History
In 1977, the Parliament of Canada gave its
Auditor a Mandate to Report whether Money
was Spent with Due Regard to Economy
and Efficiency in the Acquisition and
Management of Goods and Services and
whether the Effectiveness of Programs is
being Measured and Reported

6. More Brief History
Comprehensive Auditing is now Practiced in
Virtually all Provincial Governments and in the
Federal Government
It is now Practiced by both External, Legislative
Auditors and Internal Auditors
Federal Crown Corporations are by Law
Required to Conduct Periodic “Special
Examinations” that Invoke all the Principal
Elements of Comprehensive Auditing

7. Economy
Refers to the
Acquisition of the
Appropriate Quality and
Quantity of Financial,
Human and Physical
Resources at the
Appropriate Times and
at the Lowest
Reasonable Cost
 Right Amount
 Right Place
 Right Time
 Right Kind
 Right Cost

8. Efficiency
 Maximizing outputs
for fixed level of
inputs
 Minimizing inputs for
fixed level of outputs
 Inputs include
physical and human
resources
 Outputs include
services provided
 Measures the Use of
Resources [Outputs]
 Does not Measure
Quality or Relevance
[Outcomes]
[Efficient use of Resources
does not mean Business
Outcomes were met!!!]

9. Effectiveness
Refers to the
achievement of
objectives or other
intended effects of
operations, activities
or programs
 Highest Level of
Accountability
 Most Elusive
 Measurement often
not Reflected in
Standard Time Period
[Fiscal Year]
 Program, Process,
Service Relevance

10. Economy
EfficiencyResults
(Effectiveness)
Outcomes
Outputs
Production/
Delivery
Process
Inputs
Physical
Resources
Acquisition
Process
Money
Customer/Client Satisfaction
Client Served
Mission/Goal Achievement
Financial Viability
Profit
Cost Benefit/Cost Effectiveness
Quantity
Quality
Timeliness
Price/Cost
Unit Cost
Productivity
Quantity
Quality
Timing
Price
Unit Cost
Productivity
Policies
Amount
Timing
Value For Money Model
Source: Adapted from - Performance Auditing, A Measurement Approach 2nd Edition; Ronell Raaum and Stephen Morgan
Human Resources
Materials
Policies
Procedures
Controls
Goods
Services
Programs
Goods
Services
Input
Process
Output
Outcome

11. Private Sector Government
Example Outcome Measures
Customer Satisfaction
Market Share
Earnings
Profit
Return on Investment
Liquidity
Dividends per Share
Customer Satisfaction
Proportion - Target Population Served
Mission or Goal Achievement
Break Even; Cost Recovery
Cost-Benefit
Financial Viability
Cost-Effectiveness
Source: Performance Auditing, A Measurement Approach 2nd Edition; Ronell Raaum and Stephen Morgan

12. Value-For-Money Audit Standards
 PS 5300 of the CICA Handbook [Standards for
Assurance Engagements]
 The IIA’s International Standards for the
Professional Practice of Internal Auditing
 INTOSAI [ISSAI 3000 – 3100, Performance Audit
Guidelines]
• Value-for-Money Audit Manual [Auditor General of
Canada]

13. What is Value??
Which do you Prefer……..Value??

14. Information Technology
How do we know IT
Is Enabling Positive -
if not Transformational -
Business Value??
Value?

15. 2008 – 24% Fail, 32%
Successful, 44% Challenged
[CHAOS SUMMARY 2009]
2002
20% Of All Expenditures - Wasted!!
Where is the Value??

16. IT Project Failure = Lost Value
• Oh…That’s What You
Wanted!! [Requirements not Defined]
• Geez….I Thought Everything
Was On Track!! [Weak Project
Manager]
• It’s Those Technology Guys!!
[Business Owners not Involved]
• It Worked When We Tested It!!
[No Change Management]
• Only 1000 Days to Retirement,
I’ll Wait It Out!! [No Commitment]
• ………..and More!!
Excuses and Face Saving!!

17. IT Project Costs
20%
80%
20% [Software Costs]
80% [Project Management, Process, Bureaucracy…etc]
Some Necessary…..some not!!
The Technology is not the Problem as much as how it is Used!!

18. Value/Governance Relationship
Effective IT Governance
is the single most
important predictor of the
value an organization
generates from IT.
Peter Weill and Jeanne W. Ross – IT
Governance, How Top Performers
Manage IT Decisions for Superior
Results
Strategic Question
Architecture Question
Value Question
Delivery Question
Adapted from Val IT Framework 2.0
[Input] [Outcome]
[Process] [Output]

19. IT Governance
1. Strategic Alignment
Aligning with the Business and
Providing Collaborative
Solutions
2. Value Delivery
Focus on IT Expenses and
Proof of Value
3. IT Asset Management
Knowledge, Infrastructure and
Partners
4. Risk Management
Safeguarding Assets and
Disaster Recovery
5. Performance Measurement
IT Scorecards

20. Value and Governance
IT Governance defines a structure of relationships,
processes and measures to direct and control IT
assets (e.g. people, finance, infrastructure) in order to
achieve the enterprise's goals by adding value while
balancing risk with return
It helps to define roles and responsibilities and
specify an accountability framework to encourage
desirable behaviour in IT and accountability for the
use of IT assets. IT governance also helps to
standardize best practices and define monitoring
methods

21. Value and Governance Issues
 Heightened Management Expectations
 Linkage of Managing IT Services and
Priorities to Business Risks, and Need for
Effective Internal Control
 Best Practices - What are they, and are we as
an Organization Appropriately
Implementing?
 Just how exactly do we know if IT is being
Managed Effectively?

22. Risk and
Opportunity
Risk
Management
Value
Management
IT Governance
and Process
Management
IT Related
Events
Risk and
Opportunity
Risk IT Val IT
Cobit
IT Performance Framework
Source ISACA - 2009

23. IT Value Architecture

24. IT Architecture/Value Mapping
Funding
People
Equipment
Tools
Inputs
Processes
Outcomes
&
Outputs
Strategic Business Objectives
IT Business Objectives
IT Governance Board

25. Outcomes Corporate Profitability - Private Sector [Short and Long Term]
Program Success - Public Sector [Citizen Satisfaction] Effectiveness
Outputs Value Capture – Increased IT Profits, Increased Service Delivery
[External] Customer Loyalty and Retention – Increased Sales/Use from Existing Customers
Customer Acquisition – Increased Sales/Use from New Customers
Channel Optimization – Increased Site Traffic and Sales Efficiency
Outputs Direct Cost Savings – Reduced IT Costs and Other Direct Costs
[Internal] Improved Quality – Reliable Information, Less Inspections, Lower Cost of Quality
Increased Capacity Use – Optimal Use of Existing Resources
Time Savings – Shortened Process Cycles
Increased Productivity – Operational Improvements Efficiency
Processes IT Systems – Appropriate Processes for Effective Implementation
IT Structure – Integration into Business Unit Structure
IT Strategy – Coherent and Aligned Strategy
Leadership – Commitment and Focus on IT Initiatives Efficiency
Inputs Resources – Adequate Capital & People
Corporate Systems – Training, Processes and Culture
Corporate Structure – Organization Structure
Corporate Strategy – Alignment/Business Integration
External Environment – External Force Adaptation Economy & Efficiency
IT Performance Measures

26. D1. Recognition of Staff
Suggestions
D2. Staff Absenteeism Rate
D3. Staff Credentials
D4. Staff Retention Rate
D5. Internal Promotion Index
D6. Development Hours Index
D7. Staff Satisfaction Survey
Help Executive Management, Operation
Management and staff fulfill their
stewardship responsibilities/
accountabilities.
Resource
Management
Ensure IT resources and
infrastructure are
appropriate.
Our People/Staff
Hire, motivate, develop,
promote and retain quality
staff.
Processes Maturity
Ensure process maturity
level is appropriate for
environment.
B1. In-house vs. contract considered
B2. Lease vs. Own Considered
B3. Build vs. Buy Considered
B4. Life Cycle Costs Considered
B5. Cost of Service Measures
C1. Process Mapping & Gap Assessment
C2. Risk Assessment & Management
C3. Quality Assurance Results
C4. Customer Satisfaction Survey Results
C5. Service Levels Monitored & Reported
A1. Regular Meetings with ITSC
A2. Strategic Plan Approved
A3. Strategic Plan Updated Periodically
A4. Annual Business Plan Completed
A5. Executive Satisfaction Survey
IT Steering Committee
Ensure alignment of the IT
function with corporate
mission and goals.
Value-for-Money Scorecard for IT Services

27. Define
Objectives
Assess
Structure
Assess
Resources
Assess
Processes
Assess
Performance
Address
Issues
Business
Objectives/
Outcomes
IT Objectives/
Outcomes
IT
Performance
Measures
What Do You
Want to
Accomplish?
Strategic
Plan-Link
IT
Governance
Board
Risk
Management
Framework
Organization
Structure
Culture
People
(Capacity and
Capability
Technology
Education and
Development
Funding Technology
Acquisition
[Procurement]
Technology
Management
Project
Management
Service Delivery
Standards
Benchmarking
Architecture
Link to
Objectives
Performance
Reporting
Information
Integrity
Compliance
[Laws and
Regulation]
Validity of
Measures
Economy
Improvements
Efficiency
Improvements
Effectiveness
Improvements
Reassess
Objectives [If
Needed]
Customer
Satisfaction
Governance?
Value For Money Review Approach
Risk

28. Define
Objectives
Identify
Risks
Analyze
Effect and
Cause
Determine
Significance
and
Likelihood
Risk Assessment/Control Design Process
Method for
Managing
Risk
Design
Control
System
Business
Objectives/
Outcomes
Performance
Measures
(KPI)
Risk Appetite
& Tolerance
(KRI)
What Do You
Want to
Accomplish?
Risk
Inventory
What Can
Go Wrong
to Prevent
Meeting
Objectives/
Outcomes?
Events
Potential
Harm (What
Might
Happen?)
Opportunity?
Why Does the
Risk Exist?
(Root Cause)
The Relative
Importance
Within the
Context it is
Being
Considered
(Impact)
A Probability
or Chance of
a Risk or
Event
Happening
Inherent Risk
Avoid Risk –
(Stay Out of
the Program
or Business)
Accept the
Risk (Take a
Chance)
Reduce to
Acceptable
Level
Transfer
(Insurance)
Controls
Mitigate Risk
Controls are
Cost Effective
If there is no
Risk, there is
no need for a
Control!!
Design to
Seize
Opportunity
Management - Develop Risk Mitigation Strategies either Strategically and/or Operationally
Internal Audit - Provide Control Advice to Clients

29. Useful Tools
Val IT
• Business/IT Partnership
• IT Investment Common
Language
• Supports Better
Investment Decisions
• Potential Cost Reductions
• Supports IT Enabled
Business Change
Management

30. Useful Tools
Risk IT
• Guiding Principle
Framework for Managing
Risk
• IT/Enterprise Risk
Integration
• Risk Common Language
• End to End IT Risk
Management – Tone from
the Top to Operations

31. Useful Tools
COBIT
[Integrated with
Risk IT and Val IT]
• Improves IT Efficiency and
Effectiveness
• Better IT/Business
Integration
• Support Better Resource
Management
• Potentially Enable and
Maximizes the Business

32.  Information Technology
Supports the Enterprise
in meeting Overall
Business Goals and
Priorities
 Information Technology
does not Exist for its
Own Sake and for its
Own Ends
Thoughts

33. Paul Wallis
Director - Internal Audit
Region of Peel
Brampton, Ontario
Canada
paul.wallis@peelregion.ca
Ron Foster
Auditor General
City of Oshawa
Oshawa, Ontario
Canada
RFoster@oshawa.ca