- UK productivity is held back by deficiencies in management skills, and interventions at the firm level could potentially improve whole economy productivity. However, there is a long causal chain between interventions and impacts, and many unanswered questions remain. - Randomized controlled trials of business advice programs in the UK showed increased use of advice but were too early to measure impacts on business performance. Emerging evidence suggests businesses receiving interventions grew faster but time lags mean full impacts may take 3-5 years. - Key unknowns include impacts on productivity at the firm and economy levels, avoidance of substitution effects, and whether benefits are scalable economy-wide. High-quality evaluation of interventions is needed but has been challenging.

1. PRODUCTIVITY,
FIRM-LEVEL INTERVENTION &
HIGH-QUALITY EVALUATION
Dr Tony Moody | Business & Science Group | BEIS
October 2016
Tony.Moody@beis.gov.uk
1

2. Belief: UK Productivity is being held back
by deficiencies in management and
leadership capabilities.
• Productivity decompositions attribute much of the
productivity gap against G7 competitors to TFP, rather
than labour quality or capital investment. Management &
leadership capability would appear in TFP.
• International comparisons place the UK in mid-table for
quality of its management and leadership.
• Deficiencies in UK management skills are a key driver of
the productivity gap
• Bloom et al (2014) estimate that about a quarter of
the UK’s productivity gap with the United States
could be down to poor management
• Recent research with the ERC has expanded analysis to
SMEs in the UK and finds shows similar links between
skills, practices and performance (lower diagram), and a
‘long tail’ of firms with room for improvement
• UK Government invests extensively in improving
workforce skills (schools, University, vocational quals).
Likewise in improving capital markets, R&D, innovation.
These are the ingredients of productivity.
• But investment in how the ingredients are combined –
the recipe and the culinary skills – in UK is low,
suggesting potential for intervention to valuably improve
how people and resources are used more effectively.
How management skills and practices lead to improved outcomes
Source: Constraints on Developing UK Management Practices, BIS 2011
Source: BIS (2015) Leadership and management skills in small and medium-sized businesses.
Average management and leadership scores by country,
ranked from low to high
2

3. Hypothesis: Business-Level intervention can
Improve Whole Economy Productivity
• Adding fertiliser to a plant to improve the ecology of a habitat.
• …Yet, so many processes between intervention and impact,
and spanning big differences in scale
?
3

4. Where to start?!
Remains extremely difficult to pick future HGFs
• Predicting future high-growth prone to Type I & Type II errors
• Only know the characteristics that improve the chances of
future high growth:
• E.g. capabilities of owner/managers, level of ambition,
market opportunities.
• …but often unobservable and even then hardly high precision.
• If hard to pick winning businesses, even harder to pick proven
winning policy interventions!
4

5. Logic Model with Huge Black Box
Economy-level impacts
Displacement
(factor market
and/or product
market)?
Firm-level impacts
Perceptions or
business metrics
Proxies for
productivity.
Intervention
Selection effects? Purity of treatment
Firm to
receive
intervention
Productivity
Impact?
5

6. There is scarce robust evidence to measure
impacts or improve design
• Evaluations typically focus on the beneficiaries’ assessment of benefits and a
single policy design
• This approach does not provide a robust assessment
• The UK What Works Centre conducted a review of impact evaluations from
across the OECD
• Just 1% provided credible evidence of growth impacts
Area
Evaluations
Reviewed
Robust approach
Growth impacts
identified
Business Advice 730 23 14
Access to Finance 1450 27 17
Innovation 1700 63 9
6

7. Randomised Control Trials (RCTs) of
business advice in BEIS
• Using RCTs to address specific
questions in our hypothetical
causal chain.
• If one link in the circuit doesn’t
work, then whole black box
won’t work.
• Four RCTs on Business Advice:
- Main Growth Vouchers Programme (GVP)
- Growth Vouchers Additional Target group
- Business Schools RCT
- Growth Impact Pilot RCT
7

8. Growth Vouchers Programme - An
example of a large scale trial
• UK launched the Growth
Vouchers Programme as a
research trial testing the
benefits of supporting
businesses to use external
strategic advice
• Between Jan 2014 and
March 2015:
• 38,000 applications
• 28,000 diagnostics
• 20,000 vouchers issued,
worth up to £39.4 million
• 6,400 vouchers used,
£11m claimed
• RCT implemented
successfully
8

9. Growth Vouchers Programme – Early Evidence
• Subsidising cost of Business Advice led to increased use of
advice in treatment group (no evidence of substitution).
• Positive initial perception evidence from treatment group in
terms of future business actions needed for growth.
• But too early for business performance metrics (and take a long
time to measure in admin data because of reporting lags).
• There might be an adverse impact on self-assessment of
business skills within treatment as they are exposed to what
they don’t know, and consider focus (tackling ‘how’, not ‘what’).
9

10. Emerging lessons from wider BEIS evaluation
• Can’t pick likely future winners from data. We cannot identify future
HGFs from standard business data on current and past characteristics
(e.g. growth trajectories, size, age, sector, geography).
• …yet businesses receiving intervention grow quicker. Businesses
participating on GrowthAccelerator (part of Business Growth Service)
grew 3x faster than peers – so something going on – either successful
selection based on richer data (inc. soft data), or an indicator of
intervention impacts.
• Perception data from managers may not align to business
performance.
• How businesses act on advice varies. We don’t know what changed
behaviour will produce the most improvement.
• Time lags. Our wider evaluation evidence suggests that impacts on
business metrics might take 3-5 years to reach full effect – just as
many evaluations stop listening.
10

11. Many Questions remain unanswered
We don’t know:
• Improvement in SME-level productivity. [We measure turnover/jobs,
not GVA]
• Improvement in wider economy productivity.
But suspect:
• Rather than measure productivity directly, past interventions seek to
minimise substitution or displacement through eligibility and targeting.
• Return of investment unlikely to be scalable to offer impacts
measureable at economy-wide level.
• …but they are likely to be a helpful investment with a speedy return
because they engage currently economically active and ambitious
businesses.
11

12. PRODUCTIVITY,
FIRM-LEVEL INTERVENTION &
HIGH-QUALITY EVALUATION
Dr Tony Moody | Business & Science Group | BEIS
October 2016
Tony.Moody@beis.gov.uk
Our research, evaluations and surveys are published here:
https://www.gov.uk/government/collections/enterprise-analysis-research
12