This document discusses post-completion evaluations at VicRoads. It begins by defining post-completion evaluations as reviews of project outcomes and performance after implementation to provide accountability, best practices, learning, and improvement. VicRoads was motivated to improve evaluations following an audit and changes to its investment framework. The framework uses a plan-deliver-learn cycle and key evaluation questions on appropriateness, efficiency, effectiveness, unintended outcomes, and impact sustainability. Evaluations of projects like the Koo Wee Rup Bypass proved benefits were achieved and identified strengths and weaknesses. While evaluations provide learning, limitations include lack of data and measuring community and efficiency impacts.

1. Reviewing the Benefits:
Post completion evaluation
in planning and practice
Sarah Lowe (WSP), Reena Lall (WSP) and Sarita Narayan (VicRoads)
Presented by Sarah Lowe, WSP
AITPM 2017 National Conference

2. — What are post completion evaluations?
— VicRoads context and drivers for change
— The Investment Framework
— Evaluations in practice
Overview of main discussion points

3. Post
completion
evaluations
Narre Warren Cranbourne Road duplication

4. 4
We are good at planning –
But we can improve on evaluation and learnings from
projects.
Could do better at demonstrating that completed projects
have achieved their intended benefits, and learning from
our completed projects
The problem

5. 5
Post completion evaluations review the outcomes and
performance of an initiative after it has been implemented.
What is post completion evaluation?

6. 6
To provide information for:
• Accountability
• Best practice
• Learning and improvement
• Interest
Why do we do post completion evaluations?

7. At a glance
VicRoads
context

8. 8
• VAGO report findings
• Department of Treasury
and Finance Investment
Management Approach
Investment is “the
commitment of the
resources of an
organisation with the
expectation of receiving a
benefit” (DTF).
VicRoads drivers for change

9. 9
holistic evaluation approach relating
to Appropriateness,
Efficiency,
Effectiveness,
Unintended Outcomes and
Impact sustainability.
Organisational shift from just
reviewing
implementation or
just before and after
studies

10. 10
The VicRoads Investment Cycle
Planning for
outcomes upfront
Delivering to
maximise those
outcomes
Evaluate and learn
at completion to
improve future
decision making

11. 11
Evaluation in the Plan-Deliver-Learn cycle
Learn stage
•Carry out post completion
evaluations and reporting.
•Distribute results to
appropriate stakeholders.
Plan stage
Plan and schedule post
completion evaluations.
•get the logic of
investment right
•identify correct benefits,
indicators and targets
•consider what data is
needed to address the
KEQs
•identify stakeholders
•Allocate resources and
timelines, funding and a
commitment to
undertake a post
completion evaluation.
Deliver stage
Here the focus is to progress reporting
and monitoring (i.e. collection of data to
use in post completion evaluations).

12. At a glance
Investment
Evaluation
Framework
12

13. 13
The framework for post
completion evaluation.
It provides principles,
policy positions and
guidance for undertaking
post completion evaluation
in VicRoads
Investment Evaluation Framework

14. 14
The KEQs are designed to guide and frame the evaluation.
The KEQs are:
KEQ1 Appropriateness
KEQ2 Efficiency
KEQ3 Effectiveness
KEQ4 Unintended Outcomes
KEQ5 Impact Sustainability
What are the KEQs?

15. Evaluations
in practice
Koo Wee Rup Bypass

16. 16
Evaluations have:
• Proved benefits have been met
• Confirmed the appropriateness of projects
• Identified strengths and weaknesses
• Provided information for future projects
Results of evaluations

17. 17
Some areas which are a
challenge include:
• Lack of data or
missing benefit data
• Community feedback
• Efficiency data
• Similar project data
Limitations and challenges

18. 18
• Knowledge of evaluations
• Learning from all results, positive and negative
• Improvement in the planning part of the cycle
• Dissemination of results
Learnings at VicRoads

19. 19
• VicRoads has made great progress in post completion
evaluations
• The Framework has provided a stand-alone document to
guide the practice of post completion evaluations.
• Undertaking post completion evaluations has been useful
to assist with learning and improving
• there are still some limitations and challenges to
overcome.
• will continue to improve over time through training and
learning
Conclusions

20. Thank you
20
Sarah Lowe, WSP
Reena Lall, WSP
Sarita Narayan, VicRoads