This document discusses the importance of using forecasting methods that are "fit for purpose" rather than solely relying on "state-of-the-art" techniques, especially in rapidly developing environments like many cities in India. It notes that state-of-the-art models from Western countries may not account for the unique conditions and lack of stable historical data in developing areas. The author advocates for simpler, transparent methods that clearly explain assumptions and limitations. Overall forecasts should evaluate multiple scenarios instead of just base cases, and emphasize learning from past mistakes to improve future forecasts.

1. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 1
Foreword
Given its pace of development, India is becoming a magnet for transport
professionals. Many are keen to apply what they have learnt from overseas, to
deploy “international best practice” or “state-of-the-art” techniques. However, it
may be necessary to re-question what is most fit for purpose. Optimistic forecast
assumptions sometimes need tempering with a touch of cynicism, to address
possible risks, even if this goes against the consensus view.
Practitioners may feel encouraged to use state-of-the-art, as this is often equated to
best practice. However, such techniques are typically developed in western
environments where conditions are usually quite stable and a wealth of data are
available, going back many years. It is the combination of relative stability and
data (combined of course with research budgets) which enable innovations to be
made. State-of-the-art techniques are often embellishments to pre-existing
methods. That is not to say that such techniques do not have a role; but their
limitations should be considered.
Despite practitioners’ best efforts, forecasts all too often remain embarrassingly
inaccurate; demand frequently being substantially over-forecast. There may be
over-emphasis on the state-of-the-art rather than concentrating more
fundamentally on appropriate, fit for purpose methods. This is also true in the
West, but such dangers multiply in rapidly developing environments.
The Nature and Challenges of Rapid Development
In an urban context, rapid development typically comprises rapid economic
growth accompanied by population growth (e.g. rapid urbanisation), quite possibly
together with new urban development areas (population, commercial and/or
industrial centres). All of these can change previously existing consumer – and
transport – behaviour. The development of new transport modes can also
drastically change habits, so I would also include this in a broad definition of rapid
development; examples would be the implementation of metro systems for the first
time in a city, or an area of a city.

2. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 2
It is this destabilisation or transformation of behaviour which poses particular
risks in rapidly developing environments. Data (often relatively sparse to begin
with) can become rapidly out-of-date; and data based on previous conditions can
be misleading.
The pace of change poses challenges to institutions: can government ensure
planning codes etc are up-to-date and enforceable? Do they have sufficient staff
with sufficient skills and sufficient time to keep up?
Can transit operators cope with evolving changing demands or are they stuck
operating legacy networks attuned to how cities were, not how they are? A further
problem is one of transition: as networks evolve from ones which were viable (or at
least with manageable subsidy levels) reflecting old demands towards new
networks reflecting new demands, management and financing of the changes can
be a particular challenge.
Moreover rapid development entails far greater uncertainty. All too often forecast
assumptions of economic or population growth are based upon trends in the last
few years, extrapolated perhaps 30 years into the future. Given the effects of
compounding a small variation per year can translate into a very large difference
thirty years hence. Furthermore, rapid development is rarely linear: different parts
or segments of a city and its population will change in different ways and at
different rates.
Different Baseline Conditions
India has quite different baseline conditions from western cities, more than simple
quantitative differences in income levels, for example: labour market structures,
income distribution, urbanisation rates (including growth thereof) and urban
structures. And differences occur between Indian cities also. Vehicle mix is also
very different. In the west the vast majority of transport is motorised, with some
people now making a lifestyle choice to switch to cycling. Conditions in India are
quite different.
Such differences must be borne in mind when determining how to analyse
transport patterns to assess possible new transport infrastructure and/or policies.

3. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 3
India has a much more varied
mix of everyday modes used
for urban transportation than
do western cities within which
“state of the art” practices
have typically been
developed.
(Photographs ©Richard Di
Bona, 2012)

4. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 4
Non-motorised modes still account for a significant proportion of everyday travel in much of India, yet
trends are towards motorisation. This is in contrast with much of the West where motorised transport is
overwhelmingly the norm, with in some cities, people increasingly choosing to use non-motorised transport.
(Photograph ©Richard Di Bona, 2012)

5. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 5
But How Good Is Western Practice Anyway?
It is perhaps also worth reviewing how successful transport planners and modellers
have been in the West.
Focussing simply on toll road studies, where there is typically a single road as the
focus of investigation, one might presume that forecasts ought not to be too
unreliable. Yet experience quite strongly suggests otherwise: with average initial
year traffic on toll roads being just 70% of forecast levelsi
and rather than being
attributable simply to ramp-up, forecast errors are often quite consistent over the
first five years of operationii
.
Bias, Group Think, Agency Theory and Bidding
It is interesting to note that forecast performance correlates to who commissions
forecasts: the 70% average becomes 82% (so not so bad) when lenders commission
forecasts, but just 66% if commissioned by othersiii
. This suggests that forecast bias
is to at least an extent influenced by clients, despite the profession’s protestations
of objectivity and neutrality. Indeed, research I previously conducted found
practitioners to have only weak acceptance of bias in their work, despite
acknowledging that over-forecasts are more prevalent than under-forecastsiv
.
Planners and engineers alike are keen to create solutions and see them
implemented. And although having a study team all “buying into” helps focus
attention, there are dangers of descending into Group Think. Excessive optimism
can blind those involved to a scheme’s potential weaknesses. Team members may
be unwilling and uncomfortable to question their colleagues, bosses and friends. So
rather than identifying (and addressing) downside risks, they are overlooked (and
unaddressed).
Having formulated the “big idea” comes the challenge of getting it approved. It
may be competing against other schemes for funding (be it from government or the
private sector). There may be incentive to overstate benefits and downplay risks:
after all, a rival scheme might do that to obtain funding.
But an oft quoted statistic should be noted: every 5 years 80% of businesses fail –
so transport planners are not alone in such errors of optimism.

6. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 6
Optimism, Skyscraper Theory and Economic Cycles
Typically during (and particularly towards the end) of boom times, confidence in
an endless boom often takes hold. Politicians might proclaim that they have
beaten economic cycles (“We will not return to the old boom and bust”) as
development projects becoming aggressively more grandiose. Skyscraper Theory
observes that the world’s latest tallest building typically opens as an economic
crash engulfs its location (e.g. New York’s Empire State Building in 1931, Kuala
Lumpur’s Petronas Twin Towers in 1998, Dubai’s Burj Khalifa in 2009/2010).
These become quite visible manifestations of previous excessive optimism.
Similar problems face transport infrastructure: the following Figure shows how
projects conceived prior to a boom, opening in the early phases thereof are likely to
be relatively successful (conditions improve whilst implementation occurs). These
successes encourage more projects, as conditions continue to improve; these
likewise may be successful.
Figure 1: Interest in infrastructure projects (and their performance) across a notional economic cycle
Towards the end of a boom, the number of projects being planned increase
markedly. But owing to lead times between conception and completion, by the
time this larger set of projects are completed, economic conditions have
deteriorated. These projects are not deemed successful, leading to fewer projects
being undertaken.

7. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 7
Left: he Petronas Twin Towers, Kuala
Lumpur, Malaysia. These were the tallest
building in the world on opening in 1998,
as the Asian Financial Crisis ravaged
South East Asia.
(Photograph ©Richard Di Bona, 2010)
Left: This shows Burj Khalifa under
construction in Dubai. This became the
world’s tallest building at the time of its
opening in January 2010, as Dubai was
affected by its financial crisis.
(Photograph ©Richard Di Bona, 2007)

8. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 8
Infrastructure is Still Important
The experience of railway development in the UK provides a good illustration of
infrastructure remaining important: networks were developed privately. Often
early investors did quite well; but later investors did not. However, small towns in
UK with branch lines prospered (even if the investors in the branch lines did not),
whilst those towns without railways declined.
The problem is that infrastructure development can be hampered by perceptions
related to the relatively many projects conceived during boom times which open
after the boom has gone.
Back to those State of the Art Techniques
Models are by definition simplifications. And different models can have different
uses. The State of the Art should not be ruled out per se. Rather, it should be
understood where and how such techniques might be worthwhile.
Traffic micro-simulation can be useful at determining in more detail the likely
performance of traffic management and engineering measures. However, given its
sensitivity to traffic flows, such analysis is best used for just short-term assessment.
Forecast uncertainties should preclude this from longer-term assessments in
rapidly developing environments. Also, driver behaviour can vary a lot between
countries and between different parts of the same country (both rural versus urban
and between different cities). So such models require very careful calibration to
local conditions. Of particular concern in an Indian environment is the wide mix of
vehicle types and determining how they interact with one another.
Trip-chaining and activity-based models look at daily travel patterns and linkages
between different trip purposes; this in contrast to traditional trip-based models.
Increasingly popular amongst many (though by no means all) practitioners in
western economies, these require much more data to establish robust and
meaningful functions, compared to traditional trip-based models. Consequently,
they are quite vulnerable to error in rapidly developing environments, if relied
upon to give the “best” answer. Nevertheless, they may have a role to play if
paired with land use-transport interaction (“LUTI”) models.

9. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 9
Such LUTI models look at how land will develop (within land use zoning
constraints which are usually assumed to be strictly enforced) taking into account
transport accessibility and linkages with activity-based models. They iterate
between land use and transport models. These models can be useful at a very
strategic level to evaluate different ways in which cities may develop. However,
they too require a lot of assumptions such as how the length of behavioural lags
(how long people take to adapt to new land availability or transport options). As
such, these models could be used perhaps to set general development policy, rather
than to assess transport requirements in detail.
So What Should be Done?
Much of the above may appear confusing or even contradictory. To an extent that
is an inherent problem of trying to give broad advice: most situations have their
own exceptions to generic rules.
The key is perhaps to remember that our duty is to give the best possible practical
advice to decision makers. And perhaps the first step is to warn of the limitations
of the advice we can give. Nevertheless, to give more constructive advice, I would
suggest:
1. Keep things as simple as possible. Especially in rapidly developing
environments, the more variables and assumptions, then the greater the
scope for error.
2. Fit for purpose means best suited to the project’s requirements. Adopting
state of the art techniques for personal vanity or CV building is likely to
backfire. For sure, appreciate the range of techniques available, but the best
way to develop a profile and capabilities is to be successful through finding
the best solution. This does not necessarily require sophisticated modelling.
3. Explain any assumptions made which can be critical; also identify which
parameters are excluded from the analysis. For advice to be cogent, its own
limitations must be explained. Excluding less relevant factors based upon
the principle of Occam’s Razor is good practice, but document what was
explicitly chosen to exclude together with reasons for same.

10. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 10
4. Tailor analyses to the question at hand. Even if that means having to change
models for each project (and quite possibly cutting out as much detail as
there is new detail included). For policy level studies, LUTI might be feasible
(subject to data availability and reliability of course). Restrict highly
detailed analyses to short-term, small area situations.
5. Always evaluate alternative scenarios. Not just simply a Base Case with a
slightly different economic and/ or population growth rate to develop a Low
(Conservative) and High (Optimistic) Case: that is merely sensitivity analysis
(important in its own right but this does not constitute scenario-based
analysis). Think about possible development paths (social, economic and
land use) and transport policy regimes which are qualitatively different.
Develop a range of possible cases.
6. Think critically and sceptically. As British Philosopher Bertrand Russell
noted “we must be sceptical even of our own scepticism.” Ensure that your
group comes up with a set of conflicting outlooks on how things may
develop. Though some form of consensus is required to work together, if your
group agrees on every aspect then it is most likely that you are overlooking
some very real eventualities.
7. Rather than simply adopting a solution which seems to give the highest
revenue, ridership, NPV, IRR or whatever on your Base Case, find a family
of solutions which perform well across a range of outcomes deemed most
likely (and ideally which would also perform satisfactorily under less likely
outcomes). If needs be this may require the setting of “trigger points” to
implement some measures rather than necessarily pegging implementation to
Year 2020 or Year 2025.
8. And perhaps most importantly of all remember the adage “we learn from our
mistakes.” Learn the good and the bad from past experience, rather than
downplaying errors to convince people of one’s infallibility. These will often
be the best pointers one can get – from direct personal experience – when
seeking the most appropriate approaches and hence outcomes on our future
assignments.

11. State-of-the-Art or Fit for Purpose: What Really Matters when Forecasting?
(originally published in TrafficInfraTech, Volume 3, Issue 2, October-November 2012, pp 94-99)
Richard Di Bona 11
About the Author
Richard Di Bona is a Hong Kong-based transport consultant, working in
consultancy since 1992 on projects in over 30 countries. Holding a BA (Hons) in
Economics, an MSc. in Transportation Planning and Policy and an MBA, he has
undertaken a wide range of transport demand forecasts -- urban, interurban,
highways and public transport, multi-criteria evaluation of both infrastructure
projects and policy initiatives and peer reviews and audits of forecasts prepared by
other practitioners. He may be contacted at rfdibona@yahoo.com.
Footnotes:
iBain, R. & Wilkins, M. “Credit Implications of Traffic Risk in Start-Up Toll Facilities”, Standard
& Poor’s, September 2002
iiBain, R. &Polakovic, L, “Traffic Forecasting Risk Study Update 2005: Through Ramp-Up and
Beyond”, Standard & Poor’s, August 2005
iiiBain, R. & Wilkins, M. “Credit Implications of Traffic Risk in Start-Up Toll Facilities”,
Standard & Poor’s, September 2002
ivDi Bona, R.F. What are the Key Risks Associated with Private Investment in Start-Up Toll Road
Projects in Developing East Asian Economies?, MBA Dissertation, Henley Management College,
2006