The document discusses lessons learned from providing technical assistance to policymakers on regional development investments and policy. It shares examples of engagements in countries like Georgia, Romania, Poland, and Croatia. A key lesson is that research must be aligned with the policy ecosystem and make recommendations that policymakers can actionably implement. Researchers should understand their audience, communicate in an accessible format using the audience's language, build networks to disseminate findings, and ask questions that examine how regional development is or could be done in practice. The goal is for research to meet policymakers where they are and provide solutions rather than just identifying problems.
Lundin Gold April 2024 Corporate Presentation v4.pdf
LEARNING THE HARD WAY ON GENERATING IMPACT
1. LEARNING THE HARD WAY
ON GENERATING IMPACT
UNIVERSITY OF PADUA
Austin Kilroy, akilroy@worldbank.org
December 2021
2. What will you gain from this session?
We will focus on ‘real-world’ aspects of regional development investments and policy.
During the session you will:
• HEAR your colleagues’ viewpoints about how research meets public policy
• SEE a window on life providing technical assistance to policymakers
• FEEL SORRY FOR some hard-won lessons on generating impact
• THINK of some ideas that you might be able to use in your research.
You will need to contribute!
3. Where does research meet public policy?
• Q1: What are your recommendations to the Minister for getting more development impact from EU funds?
• Q2: How would you aim to convince the Minister about your views?
4. Where does research meet public policy?
The plot thickens….
• The Minister ++ ….
• … ++ all the other bits of the policy ecosystem for regional development.
5. •Example: Georgia
•- Identified regional
specializations and
functional economic areas
based on analysis of firm-
level data
Analytics and
diagnostics
•Example: Romania
•- Advising city of Brasov on
public-private dialog
structures
•- Assisting north-east
region of Romania on
innovation financing
•Example: Poland
•- Identifying a strategy for
investment promotion in
Pomorskie region
Targeted policy
support
•Example: Croatia
•Four-year program of
technical assistance to
Ministry of Regional
Development EU Funds,
including:
• - Analytics of regional
economy
•- Review of project pipeline
•- Regional integrative
strategy
•- Identification of high-
impact investments
Multi-year
program
Some ‘typical’ engagements on regional development in Europe from my work
BUT….. “There is a need for a long-term vision. It’s polluted by people like you coming here and confusing the leadership [with short-term recommendations].
… An economy has to be run by people who can see a way out of the situation. It’s like a war. It has to be engineered and given direction.”
Director General of SME support organization, in country X, in meeting with our team.
6. What did I learn the hard way on generating impact?
When’s the last time you changed your mind?
–and what made you change it?
"The greatest problem with communication is
the illusion that it has been accomplished."
George Bernard Shaw
“If 'seeing is believing' then
'participating is understanding’.”
-- my colleague in Croatia.
7. Who is the audience?
CAN YOU ALIGN YOUR
RESEARCH WITH…
-Policy touchpoints (legislation?
policy? investments?) “So what?”
-Policy narrative (language,
terminology, policy objectives)
-Timing & strategic opportunism
(lead-up to election, budget cycle, in
govt consultation, ahead of decision
meeting?)
•How to reach them?
MAKE YOUR FINDINGS
ACCESSIBLE…
-Map your audience (government
official, parliamentarian, minister)
-Build networks (to increase your
impact: think-tanks; blogs; public
officials)
-Communicate in the language and
format of your audience (not long
reports nor esoteric terminology).
Make it fun whenever possible
(cf. Karol’s session?)
•What can they do with it?
MAKE IT ACTIONABLE…
-Give practical relevance in the
conclusions/interpretation
-Be propositional. Policymakers
are already aware of what is wrong;
they need solutions to make it
right.
Some ideas that you can use?
Also: choosing research questions.
Look at ‘how’ regional development is/could be done?
(not only at ‘what’ to do)
Before you
heal someone, ask
him if he's willing
to give up the
things that
HIPPOCRATES
Further reading: ODI Research and Policy in Development (RAPID), e.g.
https://odi.org/en/publications/tools-for-policy-impact-a-handbook-for-researchers/
8. Concluding thoughts
• HEARD your colleagues’ viewpoints about how research meets public policy
• SAW a window on life providing technical assistance to policymakers
• FELT SORRY for some hard-won lessons on generating impact
• THOUGHT of some ideas that you could use.
How can you do your work in partnership with the people who will use it?
Editor's Notes
This is about political economy of converting research to policy impact.
Ask first If there are any preconceptions or aspirations about my PPT title upfront, and use those to adjust the presentation to their needs?
My impression is you’re a mix of PhDs and post-docs, and have quite a wide range of topics that relate to regional studies and regional development.
Anything:
Topic is relevant / irrelevant? This is about political economy of converting research to policy impact.
WB is reliable / mysterious / source of all evil in the world
I’m in the wrong workshop– I thought this was about horse-breeding
3 MINUTES
[READ SLIDE]
Who am I?
Working for WB for 10 years as staff, and 2-3 years before that as consultant. Worked in eastern and southern Africa, Europe, some parts of south-east Asia. Known as a bit of an ‘intrapreneur’ within the WB, to get new initiatives going.
PhD from MIT on Urban and Regional Economics
Worked previously in politics in London for 2 years and then for NGOs for several years in Europe and for think tank in China.
Like to travel. Also interested in social psychology.
15-20 MINUTES
It’s the year 2026. You completed your PhD/postdoc and are a well-respected academic in your country.
Your phone rings….. It’s the Ministry of Regional Development: they’re inviting you to a roundtable tomorrow with the Minister. The Minister wants to increase the economic impact of European structural funds in lagging regions.What will you recommend to get more development impact from EU funds? You’ve got 5 minutes to speak, then there’ll be a discussion.
Q1: WHAT ARE YOUR EVIDENCE-BASED RECOMMENDATIONS FOR GETTING MORE DEVELOPMENT IMPACT FROM EU FUNDS?
-> Engage in industrial transformation (but is S3 working?)– private sector development
Devolve more funds to the local level (but do they have the capacity?)– financial management
Convene a broader range of stakeholders (but how to refine and adjudicate their ideas)?-- governance
Commission additional diagnostics to test a hypothesis.
AND WHAT TO DO LESS OF:
* e.g. less undirected SME grants schemes, or more?
Q2: HOW WOULD YOU AIM TO CONVINCE THE Minister about your views?
i.e. technical expertise is not sufficient. Facts? Data? Stories? Impacts? Counterfactuals? Ongoing working relationship?
BREAKOUT ROOMS for 8 minutes. Then report back for 5 minutes.
+ Further question for thought: what do we mean by “impact”?? (paper citations? New research agendas? Change of policy? Jobs and regional development?)
RESEARCH FOCUS--
Will try to find one angle on the question, and do research first
Start from Italian point of view– use data on regional countries, e.g. Italy, and forecast development impact.
Improve administrative capacity, focus on skilled development of local people
Start research for the local region to see which part is underdeveloped
Research hits the road– forced to take a position not just review others
Take a position versus as researcher focus on pros & cons
Engage with narratives-- debate about Italy/Egy, so how to introduce your perspective on wellbeing & 15 mins cities
TECHNOCRATIC VERSUS INTERPERSONAL / CORRUPTION– FRUSTRATION WITHIN GROUP TOO
Rational approach to explaining the points– how to connect some ideas with other ideas, develop the ideas further.
Show data that was researched, SHOW RESULTS, include time period
Show real examples-- use clear evidence and simple terms. Solid data.
Policy debate is just for show-- doesn't matter for decisions?
How to break through?
2 MINUTES
The plot thickens….
National Ministries, multiple factions within the Ministry, local Mayors, regional administrations, competing departments.
All with existing priorities, anxieties, biases, neuroses, egos.
So, how to make sure that your research has an impact?
(Btw, my assumption is that you do indeed want your research to have an impact, not just to impress your colleagues and students.)
4 MINUTES
What shape typically have advisory services taken, that I have worked on?
[READ SLIDE]
IS THIS THE BEST WAY TO PROVIDE ANALYTICS AND ADVISORY SERVICES, TECHNICAL ASSISTANCE, ETC?
“There is a need for a long-term vision. It’s polluted by people like you coming here and confusing the leadership [with short-term recommendations]. … An economy has to be run by people who can see a way out of the situation. It’s like a war. It has to be engineered and given direction.”
Director General of the small business support organization, in country X, in meeting with our team.
10 minutes
Q: What did I learn the hard way on generating impact?
A: It’s all about people– and it’s messy-- led by personalities, preconceptions, arbitrary judgements, egos. The strength of argument barely counts for much compared to personal relationships, political economy, and soft power / influence.
Do you believe this?Question to provoke you: When’s the last time you changed your mind?– and what made you change it?
(take 1 minute to think about it)(probably easy to remember for something easy, like flavor of yoghurt or which restaurant to go to; but hard to remember for anything important like moral issues or viewpoint on policies).
4 MINUTES TO SHARE IDEAS / EXAMPLES / REFLECTIONS
What kind of data / information/ arguments / stories/ sentiments were involved?
(My personal story about when I became a vegetarian…..– I knew it was ethically wrong before but what swung it was a heart-felt story)
Also be conscious of whether different types of people change their mind in different ways and situations?
I’ll tell the story of exactly how and what arbitrary decisions are taken in the RAS engagement, including timeline of changes of Ministers and moving of goalposts
SHOW VIDEO _1 VIDEO_2 and VIDEO_ 3 from CROATIA RAS
Expected that PhD from MIT + 90 technical specialists would be sufficient to change people’s minds– but no. Being smart is not enough—and actually it may even be counterproductive, cultivate resentment of ‘experts’, etc.
Think about stories, engagement, relationships.
INDIAN PHOTO:
Here’s another illustration of the distance between ‘advisors’ and our clients. Do you think they will be using our insights?
So, going back to influencing the Minister:
People who share with you a difficult situation very rarely want your advice- what they want is your help to process a difficult situation or emotion so that they can create their own solution!
Things to make life easier-- not just to prove that people are doing things wrong.
Don’t try to use logic to prove that you’re right(!). Instead raise questions and ideas that provoke a sense of curiosity about how to solve a problem best. Try to put the other person in discovery mode.
Quotes:
"The greatest problem with communication is the illusion that it has been accomplished."
George Bernard Shaw
If 'seeing is believing' then 'participating is understanding'
Question: what can we do about this?– how to bring it back to what it means for researchers in regional development?
5 minutes
Question: what can we do about this?– how to bring it back to researchers in regional development?
ALIGN YOUR RESEARCH WITH EXTERNAL CONTEXT
FORMAT FINDINGS SO NON-EXPERTS CAN UNDERSTAND THEM
INTERPRET WHAT IT MEANS FOR ACTIONS AND PRACTICE.
4. Research about how to do things, not just what to do.
How do decisions get taken? What kinds of portfolio approaches are useful? How can risks be analyzed and managed?
5. Remember this is about human beings: Hippocrates quote
RECAP ON PPT
[READ SLIDE]
2 minutes
Main advice: don’t do your work in isolation from those who may use it.
Q&A – 5 minutes
[FOR ME:
In discussion / Q&A]: don’t try to correct views– just take them as alternative viewpoints, and ask for observations, thoughts, contributions, additions to solve the problems.
SDGs--