STIP Compass is a joint initiative of the European Commission (EC) and the OECD that aims to collect together in one place qualitative and quantitative data on national trends in Science, Technology and Innovation Policy (STIP). Its main data source is countries’ responses to the STIP survey, which is also jointly organised by both organisations and administered every two years. The latest dataset features close to 8 000 active policy initiatives and more than 10 000 policy instruments spanning 57 countries and the European Union, all of which is publicly available to support policy analysis and advice. This event presents and discusses the dataset along with other project developments, including new and updated thematic portals, policy evaluations and other linked data, and exploratory work around the use of large language models to query the database.

1. STIP Compass 2023
Policy data release and project updates
EC-OECD joint webinar
5 October 2023
Andrés Barreneche, Economist / Policy Analyst, Directorate for
Science, Technology and Innovation, OECD
This project has benefitted
from EU contribution
agreement LC-01968783.

2. 2
2
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Agenda
PAST PRESENT FUTURE

3. 3
3
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Past

4. 4
4
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Paper survey – typical survey response up to 2016

5. 5
5
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
• The policy data was published online, but not
easily accessible (only 2k views per year).
• The EC had a separate data collection
exercise on research and innovation policies.
• EC and OECD member countries requested to
consolidate both surveys to reduce the
reporting burden.
• Since 2015, both organisations have collaborated
to streamline reporting, data curation and
publication processes using an integrated digital
infrastructure.
• Launch of STIP Compass in 2018.
The beginning of the EC-OECD partnership

6. 6
6
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Present

7. 7
7
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
• Divided into 6 main policy areas and
one edition-specific “module” on Net
zero transitions.
• Conducted every two years, it aims to
collect data on national STI policies.
• The survey is distributed by gov’t
national contact points across
ministries and funding agencies.
• Administered under the aegis of the
OECD Committee for Scientific and
Technological Policy (CSTP) and the
EU’s European Research Area
Committee (ERAC).
The 2023 EC-OECD STIP Survey

8. 8
8
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
The 2023 STIP Compass database in figures
are registered in the Survey to report policy data
1 500 gov’t
officials
7 700 policy
initiatives
currently in force are accessible via STIP Compass, based on responses to 50 core
questions in the EC-OECD STIP Survey
describing the main concerns, discussions and guiding visions shaping STI policy
developments in countries
350 policy
debates
and the EU participated in the 2023 edition of the EC-OECD STIP Survey
57 countries
8k EC-OECD
publications
are semantically linked in STIP Compass, together with 42 000 academic articles
and 1 000 statistical indicators

9. 9
9
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
• Interactive dashboards
• 700+ different kinds of dashboards
• Country dashboards: Generate overview
documents from live data
• Search engine
• Google-like search
• Narrow down by keywords using a dedicated
innovation vocabulary
• Thematic portals
• Layers guidance and focusing on specific issues
• Download data
• Open API to download data in Excel/XML formats
Data interfaces in STIP Compass - https://stip.oecd.org/

10. 10
10
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Net zero portal
Maps STI policies that explicitly support the
transition towards net zero carbon
emissions.
Data: 1000+ policy initiatives from 55
countries and the EU, statistical indicators,
relevant publications (EC, OECD, academic,
etc.), etc.
Features
• Country dashboards
• Browse, filter and export policy initiatives
• Linked data dashboard
• Download full dataset

11. 11
11
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Use of the STIP Compass data – within EC-OECD and beyond
+175 K
PAGEVIEWS /
YEAR
161 82
200
• European Semester
• EU Missions in Horizon Europe
• European Research Area

12. The components of the new ERA monitoring mechanism
Online ERA
Policy Platform
Scoreboard
Will serve as an
umbrella tool for
sharing
information on the
implementation of
ERA Actions and
allowing
exchanges
between all actors
(i) A report on the state
of play of the
implementation of the
ERA Policy Agenda
every 18 months (EU-
level Report)
(ii) Annual reports on
each Member State’s
progress in the
implementation of the
ERA (ERA Country
Reports)
Regular reporting
Dashboard
Assesses the
overall progress at
EU level for a
limited number of
indicators in view
of realising the
ERA priority areas
(as defined in the
Pact for R&I)
Provides a more
detailed
monitoring of
the ERA Actions
and the
realization of the
ERA priority
areas at the
national level

13. 13
13
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
Future

14. 14
14
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
• We’re developing new thematic
portals
• Monitoring STI policies managed by
international organisations
• Linking complementary data
• 500+ policy evaluations
• Hundreds of country case studies
New features expected by summer 2024 or earlier

15. 15
15
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
LLM query and output
Exploratory work with large language models
policy initiatives
1) Expressing policy initiatives as vectors and filtering against a given user query.
2) Asking the LLM to answer the query based on relevant initiatives identified in 1).
User query, e.g. What are the main
objectives of policies supporting net-
zero transitions in the energy sector?
(up to 100 policies)

16. 16
16
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
What are the main objectives of policies supporting
net-zero transitions in the energy sector?
• Reduce carbon emissions by specific targets
• Improve energy efficiency
• Increase the share of renewable energy sources
• Improve energy security
• Develop and diffuse low-carbon technologies
• Ensure a just and inclusive transition
Long-Term Plan to Reduce Pollution and Energy Sector Emissions
Mission 2023 Strategy
France 2030 Decarbonising Industry initiative
Energy Efficiency Strategy 2045
Support of Energy Investment and Investigation in Business
National Energy and Climate Plan
Climate Agreement for Energy and Industry
Industry Leap Programme
Energy Innovation Programme
Industrial Decarbonisation Challenge
Just Transition Policy
National Energy and Climate Plan

17. 17
17
Directorate for Science, Technology and Innovation | Organisation for Economic Co-operation and Development (OECD) | www.oecd.org/sti | https://stip.oecd.org/ | STIPolicy.Data@oecd.org
• The experiments we’ve run so far
are promising, but require a deep
quality assessment
• Comparing insights with
previously developed analytical
frameworks
• Asking government officials and
domain experts to evaluate
outputs
• The goal is to robustly support
truthful and sound analysis
Challenges and opportunities in harnessing LLMs
Opportunities for STIP Compass
Enhanced accessibility and user-friendliness
through insights stemming from the database.
In particular:
• Further contextualise dashboards with text
summarising the data
• Regularly prepare and publish “data stories”
• Integrating a chatbot function

18. STIP Compass 2023
Policy data release and project updates
EC-OECD joint webinar
5 October 2023
Andrés Barreneche, Economist / Policy Analyst, Directorate for
Science, Technology and Innovation, OECD
This project has benefitted
from EU contribution
agreement LC-01968783.