How do you measure the success of transport programs? Use program logic.

1. THE VIEW Australasian Perspective
Prof Phil Charles
How do you measure the success of
transport programs? Use program logic.
Phil Charles is Professor of Transport at the University
of Queensland, Brisbane. p.charles@uq.edu.au
http://transport-futures.com
Measuring the success of
transport projects and
programs is poorly done.
Post-implementation monitoring
and evaluation is rarely conducted.
Transport planning and delivery
agencies are accountable to governments
and the community for effective use
of public funds. It is also in an agency’s
interest to demonstrate the success of
past investment in moving towards
desired transport outcomes, when
requesting funding for future programs.
The logic of connecting the
investment in a project or program
to the desired outcomes needs to
be carefully thought through.
Transport programs can be
developed either as a response to
transport problems or issues (bottom
up approach), or to move towards
future strategic objectives (top down
approach). The best approach is a
combination of both. To be able to
measure success, transport professionals
need tools to help them design the logic
of the programs they plan and deliver.
Logic program development has
been in use since the 1960’s, so it is
not a new concept. My concern is
that the concepts are being lost with
the turnover of professionals.
The Logical Framework Analysis, or
some variation, is used by international
funding agencies such as the World
Bank, DFID and Asian Development
Bank1 to make development aid
more effective and accountable, by
providing greater rigour in planning,
implementing, monitoring and
evaluating programs. Logic Mapping,
promoted by the UK Department for
Transport2, is a similar approach.
The basic concept of program logic is
that delivery of projects is a systematic,
sequential process: plan – manage –
review; from context (responding to
problems); to inputs (such as funding);
to actions (project delivery); to outputs
(transport infrastructure and services);
and culminating in desired transport
outcomes (reduced congestion,
improved safety, reduced emissions, etc).
The logic, based on assumptions from
past experience, is that actions such as
delivery of a transport project or service
… will result from the inputs provided,
then infrastructure and service outputs
are expected from these planned actions,
and desired transport outcomes are
predicted based on those outputs.
Each step along the sequence can
be monitored to ensure the project
is tracking to the desired objectives,
rather than waiting for everything to
be finished and some years down the
track before being able to measure
outcomes. For example, did the
inputs produce the activities and
result in the outputs predicted?
Program design starts with describing
the key transport challenges being
❝
However
beautiful the
strategy,
you should
occasionally
look at the
results
❞
Winston Churchill
faced and then going to the other end
of the program logic process, defining
the desired transport outcomes.
Using program logic the process
can be reverse engineered to develop
outputs that result in those outcomes,
and then the activities to deliver these
outputs and hence the inputs required.
It is an iterative process to match
available funding and priorities.
It should be remembered that the
‘what gets measured gets done’.
By designing the program logic and
identifying performance indicators for
each stage of the sequence of events,
then those involved in managing the
delivery of a transport program will have
a clear focus on what is important. If we
know that each critical component is
going to be measured and reported, we
will ensure they are monitored closely.
Program logic can help establish
success criteria, and also data
and information needed to assess
progress, as well as help visually
communicate the benefits of a
project to key stakeholders.
The bonus of using this approach is it
makes it easier to get projects approved!
Decision-makers can readily
understand what a project or program
is about, what it will deliver and how it
contributes to a government’s objectives,
as the logic process enables a clear,
concise summary to be presented.
If well done, the program logic
approach also gives decision-makers
confidence that assumptions and risks
have been adequately considered and
documented and that a performance
monitoring framework is available.
As transport professionals,
let us commit to measure and
report the success of our transport
projects and programs.
NOTE
1 Asian Development Bank (ABD) 2007. Guidelines for Preparing a Design
and Monitoring Framework. (http://www.abd.org/publications)
2 Department for Transport (DfT) 2010. Logic mapping: hints and tips for better transport
evaluations. (https://www.gov.uk/government/publications/logic-mapping-hints-and-tips-guide)
70 thinkinghighways.com Vol 9 No 1 Europe/Rest of the World