The document summarizes a presentation given on research engagement and impact. It discusses three key recommendations from the Stern Report regarding impact assessment for research institutions. These include allowing more flexibility in impact case studies, ensuring impact cases are linked to high quality research, and broadening the types of impact considered beyond just socioeconomic impacts. The presentation then provides an overview of an economic review conducted for the Northern Powerhouse initiative, including the context, approach, and key findings regarding performance gaps between northern England and other regions.

2. Research engagement with users and
Impact in International Business and
International Management
Frank McDonald
University of Liverpool Management School
BAM/IBNW Manchester 19 Sept 2016
1

3. Welcome
Welcome to our three speakers
Ian Drummond, Principal Research Officer,
Department for Business Energy and Industrial
Strategy
Nigel Driffield, Professor of Strategy and
International Business Warwick Business School
Richard Hindle, Director of SQW Limited
Thanks to MMUBS for hosting and especially to Heinz
Tuselmann
BAM/IBNW Manchester 19 Sept 2016 2

4. Stern Report
Three recommendations on Impact in Stern Report
1. Institutions should be given more flexibility to showcase their
interdisciplinary and collaborative impacts by submitting
‘institutional’ level impact case studies, part of a new institutional
level assessment.
2. Impact should be based on research of demonstrable quality.
However, case studies could be linked to a research activity and a
body of work as well as to a broad range of research outputs.
3. Guidance on the REF should make it clear that impact case studies
should not be narrowly interpreted, need not solely focus on
socioeconomic impacts but should also include impact on
government policy, on public engagement and understanding, on
cultural life, on academic impacts outside the field, and impacts on
teaching.
www.gov.uk/government/uploads/system/uploads/attachment_data/file/541338/ind-16-9-ref-
stern-review.pdf
BAM/IBNW Manchester 19 Sept 2016 3

5. Some tentative questions
What are the main characteristics of high quality
Impact?
How do we more effectively engage with users to
produce high quality Impact?
Who are good users to target?
How do we record Impact to justify high quality
Impact?
What type of research activities and whole body of
work (Stern)is likely to lead to high quality
Impact?
What impact on teaching will count (Stern)?
BAM/IBNW Manchester 19 Sept 2016 4

6. REF Impact
Ian Drummond
Department for Business Energy and Industrial Strategy
Ian.drummond@bis.gsi.gov.uk

7. REF Impact
Impact: 20 per cent of the overall results
Definition for the REF ‘Impact’ is any effect on, change or benefit to the
economy, society, culture, public policy or services, health, the environment
or quality of life, beyond academia.
Each submission included:
• Impact case studies. These four-page documents described impacts that
had occurred between January 2008 and July 2013. The submitting
university must have produced high quality research since 1993 that
contributed to the impacts. Each submission could include one case study,
plus an additional case study for every 10 staff.
• An impact template. This document explained how the submitted unit
had enabled impact from its research during the period from 2008 to
2013, and its future strategy for impact.

8. REF Impact
The criteria for assessing impacts were ‘reach’ and ‘significance’:
In assessing the impact described within a case study, the panel formed an
overall view about its ‘reach and significance’ taken as a whole, rather than
assess ‘reach and significance’ separately.
• Four star - Outstanding impacts in terms of their reach and significance.
• Three star - Very considerable impacts in terms of their reach and
significance.
• Two star - Considerable impacts in terms of their reach and significance.
• One star - Recognised but modest impacts in terms of their reach and
significance.
• Unclassified -The impact is of little or no reach and significance; or the
impact was not eligible; or the impact was not underpinned by excellent
research produced by the submitted unit.

9. REF Impact
Sub Panel report “The impact cases covered a very wide range of impact types,
including impacts within organisations, on national policy, on international
agreements and on the public. The high scoring impact cases typically provided
clear evidence of the reach and significance of the impact, the underpinning
research which was cited clearly met the threshold requirement of international
excellence and there was a strong narrative that described how the research led
to the impact”.
Profile type 4* (%) 3* (%) 2* (%) 1* (%) UC (%)
Impact 37.7 42.5 17.0 2.2 0.6
The weighted average impact sub-profile for the sub panel was:

10. REF Impact
Case studies
Criteria – reach and significance
Perhaps the key issue was evidence. Convincing evidence included:
• Citations in policy documents. Eg World Bank, OECD, UN etc. As well as national
government and private sector organisations, businesses.
• Actual changes in policy and practice that were clearly attributable to the research.
But problem here is demonstrating that this research was the key driver of change.
• Appointments to high level boards, etc.
• Public engagement/Press coverage.
• International reach clearly positive.
• Testimonials – probably not the most convincing evidence.

11. REF Impact
Example comments on impact case studies:
• Not totally convincing in terms of significance – impacts largely limited to
influencing debates.
• Some convincing details of how worked with XXXXXXXXX and XXXXX.
Membership of Advisory Group also positive. But not clear how original this
research was.
• Significant impact in one large firm, but nothing wider.
• Active dissemination – practical guidance materials for UK manufacturers,
workshops etc. But weak on actual impacts.
• Clear strategy for engaging non-academics and clear pathways to impact.
• Some of the projects cited are very small scale.
• Citations in policy documents, Advisory group membership, ongoing funding
stream. Relatively strong in terms of both reach and significance.
• May be more actual concrete impacts to come, but to date both reach and
significance limited.
• No apparent strategy for wider dissemination.
• Impacts may well be significant, but the case isn’t well made.
• Impacts exaggerated.
• Significance probably overstated – cited strategy document drew on wide
ranging sources.
• At face value has clear impacts, but lacks evidence.

12. REF Impact
Impact templates
Sub panel report … “In general, the impact templates were rated lower than the
impact cases and it seemed that some institutions were uncertain what to say in
this section of their submission. A particular problem was that some submissions
merely used the final section of the template (Relationship to case studies) to
summarise each impact case study rather than to relate them to the historic or
current strategy of the unit”.
“The extent to which the unit’s approach described in the template was conducive
to achieving impacts of ‘reach and significance”’
Not really sufficient to say we have a research services office or an ‘engagement
officer’.
Dissemination isn’t impact. But relatively few case studies demonstrated that the
research had had an integral dissemination strategy.
Need to demonstrate strategy, support and actual links to research activity/case
studies.
Culture of securing impacts.

13. REF Impact
1. Read the exam question
ensure research is eligible, ensure impact is eligible
explain any anomalies (e.g. researchers moving during REF period)
make it easy for the panel!
2. Tell a good story
be clear on contribution of submitting HEI to research (e.g. in collaborations)
explain research clearly
link impact to research
do not overstate the impact – be realistic
be clear on contribution of submitting HEI to impact
3. Provide appropriate and compelling evidence
evidence should offer diverse mix of independent qualitative and quantitative sources that directly support claims being made
follow through the ‘stories of change’ (i.e. what changed as a result of the research)
4. Communicate reach and significance of impact
describe context in which the impact takes place to give an indication of its significance
set out the original objectives of the research e.g. what was the intended reach? If you maximised the possible reach of your
research then you were likely to score higher
5. Appropriate use of language and presentation
make narrative coherent and present a linear, chronological story of development
case study should be written so it is understood by a reader without specialist knowledge
clear presentation makes it easy to read: sub-headings, adequate spacing, pictures or diagrams were well received.
make sure the case study is self contained
http://www.bulletin.co.uk/31441/research-impact-case-studies-tips-success-assessors/

14. REF Impact
Securing impact from research conducted for government
Government research is almost always commissioned to inform policy development so
should be some impact. However, can be very difficult to establish a direct link with
policy development in practice. There is normally a body of evidence and internal
documentation may well not be available.
• Set out the policy relevance of the research in the report.
• Offer to present the findings to senior officials/ministers.
• Ask whether and how the research findings are being used.
• Track citations, etc. in official reports.
• If appropriate, work with departmental Press Offices.

15. IMPACT (and how to achieve it)
– a personal perspective

16. Franks questions
What is high quality impact
How do we more effectively engage with users
Who are good users to target?
How do we record impact
What type of work leads to impact?
What type of impact leads to teaching ?

17. Main message
VCs like impact and engagement more than you like they do
WHY ?

18. Disclaimer – I first wrote the next slide in 2010!

19. Be instrumental for a minute
Research relies on funding
Universities are under pressure “to show what they are for”
So are business schools !
So are Govt Departments, HEFCE etc
So are RCUK
People who do this will be paid more, promoted faster and be
in demand in the job market (and not just in the UK!)
And try writing an ESRC bid without an impact plan – its
impossible !

20. IMPACT, ENGAGEMENT AND INTEREST
It has opened doors for me, locally and nationally, in terms of
policy engagement
I find it interesting
It has led to other funding opportunities
(most importantly ?) My VC likes it …..

21. Local Engagement
Universities feel the need to show local
engagement :
More than at any time in the last 20 years,
more policy makers seem interested in
universities:
Plus : interventions lead to evaluation and
there is money in evaluation !

22. The Private Returns
There is research money in this but it is a long journey
However – being instrumental, it is a good story to tell at
interviews – it impresses VCs
So there is a tradeoff – how many more papers could I have
written (and does it matter ?)

23. I went for a job interview…..

24. More generally…
Local engagement now needs to be seen in the context of
localism, and the contribution that universities can make to
economic growth.
Universities are keen to show this, directly and indirectly (future
funding ?)
The LEP’s have some money but this has to fit their agenda
(smart specialization)?

25. How to start this process
Matched funding - potential sources ?
How to start out ?
All city councils have no money, and oodles of great data, on all
sorts of things ?
Are you in an area where you can write papers on local data –
perhaps not for JIBS, but Economic Geography for eg ?
Industry – parliamentary trust – does your HEI work with them
?
Do you want to do media ?

26. ESRC and REF define impact differently
REF “reach and significance”
ESRC – engagement and influence

27. http://www.esrc.ac.uk/research/evaluation-and-impact/what-is-impact/
NB PATHWAYS to impact – ie who are your stakeholders and are they
really engaged ?

28. How I think about impact
Lancet :Brownell & Roberto http://dx.doi.org/10.1016/S0140-6736(14)62397-7

29. My strategy: My impact is local

30. Parochial ?
Area of WMCA is about £180bn – so not so tiny to be of no
interest
I stick to what I know – inward investment and strategies for
growth
I always have some IB theory at the back of my mind (motive?)
YOU define this – so stick to your area of interest

31. Can I publish from this work ?
To be honest probably not –
Or to be honest not in a journal that I am interested in for REF
– but I do send stuff to people like Ian and they are interested.
Equally, a paper that is a 2nd round R&R at JIBS started off as a
piece of work on local issues – in terms of the ideas.

32. Media ?
Do you want to be on the telly all the time ?
Or spend your life on twitter ?
I did this over Brexit (basically pushing a paper we had written
on Brexit and FDI)
Was picked up by various cabinet members (but we lost and
its exhausting)
I now have a set of MPs / cabinet ministers / BIS people whom
if I write something of interest I send a 300 word summary to –
who knows……

33. Routes within your university
Media office always after stories
TheConversation
Impact Accelerator Accounts ?
Associate Deans / PVCs for engagement more open to junior
people than you think – they have jobs to justify too
VCs want regional engagement

34. Finally … impact and teaching
I teach MBAS
MBAs have non-satiation of examples…..

35. Research and analysis to inform policy - the Northern
Powerhouse Independent Economic Review
BAM/IBNW, 19 September 2016 – Richard Hindle

36. Content
1. CONTEXT
Focus of the Independent Economic Review
Approach to the work, and process
2. OUTPUTS: specific findings on
The North’s ‘Performance Gap’
‘Capabilities’ of the North
Growth scenarios for a future Northern Powerhouse (NPh)
Timing
SQW team began work in October 2015
Interim outputs, briefings, presentations from December 2015
Draft and Final Reports, March-April 2016
Launch (Liverpool) – July 2016
1

37. CONTEXT (i)
Aim/objective
> An Independent, & robust Economic Review of the North’s economy
> Commissioned by Transport for the North (TfN) on behalf of partners
Three drivers…
> Data & evidence to support specific NPh transport proposals (Mar ‘16)
> Starting to frame the NPh ‘narrative’, based on this evidence
> Analysis to provide a bedrock on which to build further work
And three aims, to
> Characterise NPh’s economic position, & underpinning causes
> Take an holistic spatial perspective: transport links, cities, other areas
> Identify the ‘peaks through the clouds’, opportunities for the North
It was not about
> NPh strategy, action planning or governance
> Ruling in/out local thinking/activity
2

38. CONTEXT (ii)
Starting point - this is not understudied territory!
Economic scale, functions and geography well-attested
Many economic actors; wide range of functional links
Footprint for the North of England
16m population, 7.2m jobs, £300bn GVA
Eleven LEP areas, spanning conurbations, cities, old industrial
areas, market towns, extensive rural areas
& c.8,000 pages of evidence!
3

39. 4
Start-Ups &
Spin-Outs
In-/out-
movers
Firms & sectors
GVA
Large firms
• Congestion
• Travel to Work
• Quality of life
• Housing
• Job satisfaction
• Location
• Flexibility
• Cost
• Transport &
Connectivity
• Environment
• Culture/Ambition
• Education
• Training
• Entrepreneurship
• Access to work
Land, property,
infrastructures
Knowledge
Money & External
Business Expertise
PeopleSustainability?
• HE/FE
• Research institutions
• Strategic alliances
• Informed networks
• Banks /Angels
• Venture capital
• Accountants
• Lawyers
• Marketing Experts

40. Process – 5 workstreams
5
WS 1
Northern performance gaps:
Prosperity & Productivity
WS 2
Bottom-up place analyses
(x11 LEPs)
WS 3
Northern sectoral competitive
advantages . . . & ‘Capabilities’
WS 4
A transformed NPh: growth
scenarios
WS 5
Proposals for Independent Panel
Nov 15
Feb 16
May 16 Final Reporting
Testing & calibration with:
• Leaders & LEP Chairs
• TfN’s Partnership Board
• TfN’s Executive Board
• Economic Reference
Group (Ec Dev Leads from
all 11 LEPs)
• TfN’s Project Team
. . . & wider stakeholders, e.g.
HMT, BIS, UKTI, DfT, NIC,
CabOff, IPPR, N8, NetRail,
HS2 etc.

41. WS1: What is the ‘performance gap’…What should
policy-makers be targeting?
‘Prosperity gap’
Measured as GVA (incomes – ‘profits & wages’ - generated in the
North) per capita
Can be broken down into
> ‘productivity gap’: GVA per worker
> ‘employment rate gap’: Share of workers in working-age population
> ‘age structure gap’: Share of working-age population in whole popn.
‘Growth gap’
Measured as growth in GVA, jobs, &/or population
For infrastructure planning, it’s growth in GVA, jobs &
population that ultimately matters
But supporting argument for faster growth may be based on
closing the prosperity gap
6

42. A persistent prosperity gap (GVA/capita) over time
Marked
improvement in
the decade to
2007, partly
reversed since the
recession
Gap to European
comparators is 30-
35%
7
Rest of England
(excl. London)

43. WS1: In summary…
8
North’s ‘performance gap’ (GVA per capita) ⇨ persistent & entrenched
25% vs rest of England, or 10-15% excl. London.
Productivity: key factor explaining
gap, widening post-recession.
Contribution to gap (2009/13): 17% vs rest of England
Employment rate: gap persistent,
but largely stable over past decade
Contribution to gap (2009/13): 5% vs rest of England
What is driving
the North’s
performance
gap …?
… & what is
driving the
productivity
gap?
North’s skills gap ⇨ most important
factor driving the ‘performance gap’,
influences productivity & employment
rate.
Other important causes of the
productivity gap include under-
investment (widened notably since
2008), low enterprise rates,
agglomeration & poor
connectivity.
Sectoral mix less important.
Also, jobs per worker - contribution to gap (2009/13): 4% vs rest of England

44. WS2&3: What could we research and assess –
sectors, capabilities and competitive advantage?...
Build understanding through
Analysis of historic prosperity & productivity gaps (WS1)
Bottom-up mapping of sectoral priorities for 11 LEPs (WS2)
Top-down Northern GVA, employment & productivity forecasts -
for 45 sectors (WS3)
9

45. …and where were the main Challenges?
1. Data provide only a partial picture
Imperfect view of sectoral activity
Little intelligence on relationships, synergies & connections…
…Even less about underpinning drivers of change (market & tech)
2. Little will be gained by ‘us too’ sectoral approaches
Instead, look to Smart Specialisation thinking
> ‘Embeddedness’, ‘relatedness’, ‘connectedness’
> ‘Staying ahead’ & ‘Routeways to excellence’
3. NPh’s approach is not about ‘one sector per area’ thinking
Aim to understand ‘Peaks through the clouds’ at the level of the
North
10

46. Pan-Northern Capabilities: basis for assessment
‘Capabilities’ assessment for NPh based on clear
evidence for
Sector-based specialisation – GVA and employment
‘Hard’ assets eg international class R&D facilities & Centres of
Excellence)
‘Soft’ assets eg nationally/internationally recognised
collaborations, networks, & interrelationships
11

47. Pan-Northern Capabilities: Prime and Enabling
Four distinct ‘Prime’ Capabilities at level of the North
Advanced Manufacturing – materials & processes
Energy – technologies underpinning generation, storage, low carbon
Health Innovation – Life Sciences, Med Tech/Devices, e-health &
service delivery processes
Digital – computation, software design/tools, data analytics,
simulation/modelling
Three ‘Enabling’ Capabilities, supporting the growth and
development of the ‘prime’ Capabilities…
Financial & Professional Services (FPS)
Logistics
Education (primarily HE)
. . . all underpinned with a unique offer
Quality of Life
12

48. WS2&3: In summary
13
Advanced
Manufacturing
– materials &
processes
Health
Innovation –
Life Sciences,
Med Tech/Devs,
& service
delivery
processes
Energy –
generation,
storage, & low
carbon
technologies
Digital – computation, software design/tools, data analytics,
simulation/modelling
Financial & Professional Services
Logistics
Education (primarily HE)
. . . & Quality of Life

49. WS4: Scenarios for future growth
Key questions
Based on productivity, sector, & capability findings
> What could growth look like in the North?
> What transport & other factors will be needed to realise this growth?
> What options are realistic, on the balance between GVA &/or jobs?
3 scenarios
1. The future is like the past (‘business as usual’)
2. The sum of the ambitions set out in LEPs’ Strategic Economic
Plans, as collated by TfN (‘SEPs’ expectations’)
3. Transformational trajectory for the North’s future performance,
relative to the past (‘transformational’)
14

50. WS4&5: In summary
Under the Transformational Scenario:
At 2050, potential for…
> GVA to be 15% higher than ‘business as usual’ projection
– In 2050, GVA is £97bn higher (in 2015 prices) in the ‘transformational’
scenario than in the ‘business as usual’ case
> 850k additional jobs compared with ‘business as usual’ (& 1.56m
more than in 2015)
> Productivity to be 4% higher, with progress towards closing
productivity gap cf rest of England.
Growth led by ‘Prime’ & ‘Enabling’ capabilities, including but
not limited to agglomeration effects for FPS
Jobs growth mostly in Services
Requires improved inter-city rail links, global connectivity (air
& ports), intra-city public transport, & smart ticketing
With concomitant improvements in Education & Skills,
Graduate Retention, Innovation, Inward Investment
15

51. Next steps for the Northern Powerhouse?
Successful launch – well-received on the day, afterwards
IER part of wider work on policy development, priorities
Earlier ‘One North’ report, 2014
Publication of The Northern Transport Strategy, Spring 2016
On-going TfN work - Integrated Transport Strategy
Key question: Commitment of all parties going forward
Momentum maintained with new Minister
Last week’s announcement: Northern Ports Association…
…and George Osborne’s Northern Powerhouse Partnership
Agenda for NPh Manchester conference, February 2017
Includes Productivity, Infrastructure: also, housing, ‘big data’
With focus on implications Post-Brexit - international positioning
16