The document outlines the importance of transparent, evidence-based decision-making in the Canadian government, as emphasized by Prime Minister Justin Trudeau. It discusses the challenges faced by the Treasury Board in using data effectively for program evaluation and accountability, highlighting issues with data quality, access, and the administrative burden of compliance. The document proposes a 'metrics agenda' to improve standardized measurements and analytics, ultimately aiming to enhance decision-making and transparency in government operations.

1. 1
Delivering on Results – Evidence-Based
Decision Making through Better Metrics
Organisation for Economic Co-operation and Development
November 2015
UNCLASSIFIED
Kiran Hanspal
Executive Director - Planning, Performance, and Evaluation
Treasury Board of Canada Secretariat

2. Current Context
2
UNCLASSIFIED
The newly elected Federal Government’s platform signalled that
transparent, evidence-based decision-making will be integral to
how Canada will be governed
- Prime Minister Justin Trudeau
‘‘
’’
We will make decisions
using the best data
available and will invest
only in programs proven
to offer good value.
‘‘
’’
Responsible governments
rely on sound data to make
their decisions. We will
release to the public key
information that informs the
decisions we make.

3. 3
SECRET
Advice to Ministers
The President of the Treasury Board (TB) has been tasked by the Prime Minister with
leading efforts to:
• Ensure that departments and other federal organizations are able to use the best available
information; and
• Strengthen the culture of measurement, evaluation and innovation in program and policy
design and delivery
Within Canada’s overall Expenditure Management System (EMS), TBS supports the
TB Cabinet Committee in effectively allocating spending in a manner that ensures
efficiency, effectiveness and accountability by:
• Establishing and monitoring adherence to TB management policies (financial & non financial);
• Supporting TB approval of detailed operational plans and recommendation of resource
appropriations for new programs ; and
• Supporting TB determination of resource needs and investment opportunities for existing
programs
The EMS relies on sound performance data to provide TB Ministers with the
information required to make good decisions
• However, this data is often incomplete, inconsistent and not focussed on real results for
Canadians
3
Role of the Treasury Board Secretariat
(TBS) and Mandate of the President
UNCLASSIFIED

4. 4
SECRET
Advice to Ministers
4
Four Decades of Evidence Based
Performance Management
• New Government - Emphasis on evidence-based decision-making, transparency, and accountability
• Deficit Reduction Action Plan – Moving the government towards budgetary balance
• Financial Crisis – Fiscal stimulus
• Cyclical spending reviews – Controlling spending which had grown since Program Review
• Federal Accountability Act 2006 - Cyclical evaluation of all transfer payment programs
• Management, Resources and Results Structures – Detailed program level knowledge
• Management Accountability Framework – Assessing management performance across government
• Canada’s Performance – Linking programs to societal performance
• Results for Canadians – Results-based management as a stated priority of government
• Transfer Payment Policy – Performance frameworks and evaluations reviewed by TBS
• Modern Comptrollership – Investment in management practices and controls
• Program Review – Dealing with a large deficit through major expenditure cuts
• Improved Reporting to Parliament– Moving to results-based plans and reports on performance
• Planning Reporting and Accountability Structures – Unsuccessful try at linking resources to results
• First Program Evaluation Policy – Government-wide implementation
Late
70’s
Mid
90’s
2000
Mid
2000’s
Last Ten
Years
2015
UNCLASSIFIED

5. 5
SECRET
Advice to Ministers
5
Current Strengths
• Management Resources and Results Structure (MRRS) – presents a well-
established and effective mechanism for parliamentary reporting and
public accountability
• Reports on Plans and Priorities (RPP) – Forward Looking
• Departmental Performance reports (DPR) – Retrospective
• Program Alignment Architecture (PAA) & Performance Measurement
Framework (PMF) – provide a coherent picture of government
programming and of how each program’s expected results are organized
to achieve a department’s mandate
• Complementary measuring and performance reporting processes –
Estimates Process, Evaluation Process and Management Accountability
Framework Process
UNCLASSIFIED

6. 6
SECRET
Advice to Ministers
Quality
• Varying quality of indicators and data – including the lack of standardization – have
hindered assessments of program effectiveness and efficiency
• Since 2009-10, nearly half of all evaluations have noted insufficient data to assess
program effectiveness and efficiency
Access
• High quality, integrated financial and non-financial data and information is not readily
available or accessible to key decision-makers (TB, Deputy Heads)
Utility
• The existing PAA/PMF structure has been effective in terms of its utility for reporting
but it has been less useful for decision-making due to a lack of relevant metrics
• Large proportion of proposal for new funding do not provide adequate plans for
measuring performance and are not sufficiently evidence-based
Administrative and Reporting Burden
• The present requirements for measuring and reporting require a significant
investment in time and resources
• The perceived value of measuring and reporting is not commensurate with this
investment
6
Opportunities for Improvement
UNCLASSIFIED

7. 7
UNCLASSIFIED
7
Focusing on the whole
The majority of annual
program spending occurs
without detailed Treasury
Board scrutiny
More Comprehensive Tools Are Required to Build on
Our Strengths, and Capture Opportunities
We need better tools for …
• credible, regular assessments of key aspects of performance
• measuring and improving the back office (internal services)
• comparing performance across disparate organizations and types of programs
• facilitating the use of performance information in decision making
• transparency and increased access to performance information
$6.9Bn
Closely Scrutinized
by TB*
*2013-14 Supplementary
Estimates (A, B, C)
$232.5Bn
Not Closely Scrutinized
by TB**
*2013-14 Main Estimates
** 2013-14 Statutory Spending

8. 8
SECRET
Advice to Ministers
8
‘‘
’’
Leveraging and creating relevant information to
support sound evidence-based decision-making,
accountability and transparency … founded on
benchmarks, advanced costing methods, and
performance metrics
The Vision
• Enhancing the, utility and accessibility of information products
which describe government operations
• Improving the reliability and standardization of key underlying
data and information
UNCLASSIFIED

9. 9
SECRET
Advice to Ministers
9
Three Key Elements to Our Approach
1. Metrics
• What we mean:
Fundamental data and
information (e.g., $,
personnel, time,
inputs/outputs, etc.).
• Standardized costing
and standard measures
for efficiency,
effectiveness, and
service
2. Analytics
• What we mean:
Information derived
from metrics (e.g., roll-
up indices, correlations,
etc.), and tools for
comparing like with like
(benchmarking).
• For departmental
management, and the
budget office
3. Information
Products
• What we mean:
Incorporating the
metrics and analytics
into existing data
systems and tools
which support decision-
making
• Extending the breadth
and depth of routine
oversight within
departments, and by
central agencies of
government
UNCLASSIFIED

10. 10
SECRET
Advice to Ministers
Moving forward with a ‘Metrics Agenda’
10
Standardized
metrics
Outcomes
EfficiencyService
Standards
< ------- All Government Programs ------ >
Other programming
types
Services to Citizens
(including Transfer
Payments)
Internal Services
Functions
Standardization work is well advanced on back office/internal services
functions, and is being followed-up for service-oriented programs, including
those which administer transfer payments
Applied to …
UNCLASSIFIED

11. 11
SECRET
Advice to Ministers
Standardized Metrics
Internal Services and Services to Canadians
(including Transfer Payments)
For Back Office/Internal Services:
• Guide on Internal Services Expenditures: Recording, Reporting and Attributing which is
being modified to better support comparison of like-with-like
• Updated draft standard performance measures for 7 of the 10 categories of internal
services for implementation in fiscal 2016-17, and with an initial emphasis efficiency
measures
• operating cost of the function / operating cost of the department
• personnel delivering the function / personnel in the department
For Service to Canadians:
• Developed two standardized efficiency measures and one timeliness measure for
significant services to citizens (including transfer payments)
• Phased government-wide implementation of measures with the most materially-
significant organizations scoped for implementation in fiscal 2016-17
11
UNCLASSIFIED

12. 12
SECRET
Advice to Ministers
The ‘Metrics Agenda’ is part of our increasing
focus on evidence-driven decision making
Manage operations better - Better context on departments’
operating situations = better assessment/challenge of their
spending proposals
Shape government-wide policy together - Everyone has the
same starting point, the same snapshot of current conditions
Take stock of enterprise-wide trends - Impact of specific
measures or initiatives at the government level can
sometimes be best identified and understood through
graphical presentation styles
12
UNCLASSIFIED

13. 13
SECRET
Advice to Ministers
Contacts
• Expenditure Management Sector (EMS)
mrrs-sgrr@tbs-sct.gc.ca
• Office of the Comptroller General (OCG)
fin-www@tbs-sct.gc.ca
UNCLASSIFIED
13

14. 14
14http://www.tbs-sct.gc.ca/ems-sgd/edb-bdd/index-eng.html#start
TBS InfoBase provides a quick and easy access to detailed information on government spending
and people management
Annex A
TBS InfoBase
UNCLASSIFIED