OECD workshop on measuring the link between public procurement, R&D and innovation. "Demand side vs. supply side technology policy: hidden treatment and new empirical evidence on the technology policy mix".

Demand side vs. supply side technology policy:
hidden treatment and new empirical evidence on
the technology policy mix
Marco Guerzoni and Emilio Raiteri
Department of Economics and Statistics “Cognetti de Martiis”, University of Turin
OECD Expert Workshop, Paris, Dec 5-6
Marco Guerzoni and Emilio Raiteri Innovative public procurement and R&D Subsidies

Introduction
Aims
In this work we aim at analyzing the simultaneous effect of technology
policies (public R&D subsidies, tax credits, and innovative public
procurement) on firms’ innovative behavior.

Introduction
Aims
In this work we aim at analyzing the simultaneous effect of technology
policies (public R&D subsidies, tax credits, and innovative public
procurement) on firms’ innovative behavior.
Results
Strong result: without controlling for simultaneity of other policies,
the impact of a single policy is overestimated.
Weak result: the impact of a mix of policies is stronger than the
sum of the effects of the policy in isolation.

The impact of public R&D subsidies
Do public subsidies displace private efforts in R&D, simply add to them
or favor their increase?
Long deabated question in the literature with no conclusive answer
Some evidence of a crowding out effect (Shrieves, 1978; Carmichael,
1981; Higgins, 1981)
Some evidence of a reinforcing effect (Holemans, 1988; Link, 1982;
Antonelli, 1989 ).
Reviews of the econometric evidence
Garcia-Quevedo(2004) does a review of 74 results: Complementarity
vs. substitutability: 38-17 (19 insignificant).
David et al. (2000) discuss methodological issues: different source of
endogeneity.

The impact of public R&D subsidies II
Selection bias
Selection bias may have a dual cause:
Public institutions may cherry-pick winners on the basis of some
peculiar characteristics (e.g. more innovative firms)
Firms may possess information/search capability advantage (e.g.
larger firms may have devoted staff)
Literature addressing the selection bias issue:
Almus and Czarnitzki (2003) design a quasi-experimental setting,
applying propensity score matching to reduce selection bias
Rejects the crowding out hypothesis and provides some evidence of
a positive impact of R&D policies on private investments.
Thereafter numerous works applied the same methodology.

The impact of public R&D subsidies III
Possible drawback in this literature:”hidden treatment” as a confounding
factor
Well known problem in clinical research:
Various compounds are administered to a patient at the same time,
as a cure fore the same disease.
Is it then possible to evaluate the effect of a specific treatment
without considering the others?
In our case a hidden treatment can arise from the presence, in a
system of innovation, of other technology policies designed to
stimulate private R&D.
Suitable suspects ! R&D tax credits and Innovative public
procurement (IPP)

Innovative Public Procurement
IPP as a technology policy
Def: The purchase of new technologies, innovative products and
services by public institutions
Geroski (1990), Dalp´e (1994), Edler & Georghiou (2007),
Edquist&Zabala(2012): need for demand oriented innovation policy
Barcelona Strategy, Europe 2020 Flagship Initiative: Innovation
Union (European Commission, 2010), Oecd (2011)
Empirical evidence:
Lichtenberg (1988): public procurement has a positive effect on a
firm’s propensity to engage in R&D.
Aschhoff and Sofka (2009) find robust impact of IPP and no impact
for R&D grants on innovation output

Research questions
Summing up:
Literature investigating the impact of R&D subsidies on private
investments in R&D, controlling for selection bias, finds a
reinforcement effect (Almus and Czarnitzki, 2003).
Theoretical and limited empirical literature on IPP suggest a robust
and positive impact of IPP on private investments in R&D.

Research questions
Summing up:
Literature investigating the impact of R&D subsidies on private
investments in R&D, controlling for selection bias, finds a
reinforcement effect (Almus and Czarnitzki, 2003).
Theoretical and limited empirical literature on IPP suggest a robust
and positive impact of IPP on private investments in R&D.
Our goals:
To test the robustness of the results on the impact of R&D subsidies
when also innovative public procurement is taken into account.
To provide new empirical evidence on the effects of innovative
procurement and of its interactions with other technology policies.

Data and method
Data
Cross-sectional dataset from Innobarometer on Strategic trends in
innovation 2006-2008 survey, conducted in the 27 Member States of
the EU, Norway and Switzerland.
The project surveyed 5238 companies with more than 20 employees
in a large selection of sectors.
Senior company managers responsible for strategic decisions making
of 5238 company were interviewed
Firms have been asked about any public procurement contracts they
have been awarded and whether this procurement was innovative or
not.

Empirical strategy
Quasi-experimental framework:
R&D public subsidies, tax credits, and IPP as treatment variables.
Increase in private R&D expenditures as the measured outcome.
Propensity score matching to mitigate the endogeneity problem.
The average treatment effect will be recovered by taking the
difference in averages between treatment and control group.
Data problems.

Empirical strategy
Procedural confounding
To tackle this issue we design 10 different treatments:subsidies, tax
credits, IPP, subsidies only, credits only, IPP only, 3 treatments for
any combination of two of them, all of them.
First 3 treatments do not take into account potential policies
simultaneity ! procedural confounding. Each has different control
group.
The other 7 treatments take into account possible interactions !
eliminate (reduce) potential confounding. One control group.
Research questions
Are estimations retrieved for the first three treatments going in the
same direction along the evidence provided so far?
If yes, is there a significant difference in the estimates recovered
when reducing the procedural confounding?

Treatment and controls
Tabelle : Loss of observations due to common support and caliper
requirement
Treated group Control group Loss & Caliper (%)
Treatment vulnerable to confounding
Policy Subsidies 1141 3863 81 (7.0)
Policy Procurement 573 4453 44 (7.6)
Policy Tax Credits 1113 3789 91 (8.1)

Treatment and controls
Tabelle : cont.
Treated group Control group Loss & Caliper (%)
Treatment in isolation
Policy Subsidies only 473 2804 29 (6.1)
Policy Procurement only 284 2804 21 (7.3)
Policy Tax credit only 492 2804 41 (8.3)
Simultaneous treatments
Policy Sub Tax 414 2804 29 (7.0)
Policy Sub IPP 86 2804 7 (8.1)
Policy IPP Tax 75 2804 5 (6.6)
Policy All 86 2804 6 (6.9)

Propensity score matching estimator results
Difference in the % of firms increasing R&D
Sample R&D grants T.credit IPP R&D Only T.credit only IPP only
ATT 0.069*** 0.064*** 0.112*** 0.029 0.043 0.066*
Unmatched 0.088*** 0.107*** 0.155*** 0.35 0.07*** 0.084***
Sample R&D + t. credits IPP + t. credits R&D + IPP all policies
ATT 0.107*** 0.267*** 0.173*** 0.147**
Unmatched 0.112*** 0.310*** 0.257*** 0.270***

Main findings
Main findings:
Results retrieved from the treatments vulnerable to confounding
effect are coherent with the evidence provided in the literature.
When we consider policy tool in isolation, the positive impact of
R&D subsidies on R&D investments reduces and ceases to be
significant ! IPP proves to be a crucial confounding factor and to
have robust impact on innovative inputs.
If consider in isolation, IPP is still significant.
The convexity of the policy tools ? ! evidence points in that
direction but possible selection bias

Conclusions
Policy perspective:
Past results have been overestimated.
Our results recommend to carefully consider the interaction among
different tools in composing technology policy mixes.
Data collection for evaluation should consider this issue.
IPP may represent an effective way to foster firms’ innovative
activities and to reinforce positive effects of R&D subsidies,
stimulating additional expenses in R&D.
Push and Pull strategies should be considered at the same time.

Thank you for your attention![marco.guerzoni@unito.it]

Propensity score distributions before and after matching

Empirical strategy II
Selection bias
To reduce the selection bias we follow Almus and Czarnitzki (2003)
who brought in innovation policy studies non-parametric matching.
We apply propensity score matching (Rosenbaum and Rubin, 1983)
to find a group of non-treated individuals that are similar to the
treated ones in all relevant pre-treatment characteristics.
Once the matching is implemented treated and control units should
be on average observationally identical. An unbiased estimation of
the average treatment effect should be retrieved by:
PsmATT = Ep(x)|T{E[YT|T, Pr(X)] − E[YC|C, Pr(X)]} (1)

Propensity score specification
Estimation of the propensity scores
To recover the propensity score for every treatment we run different
probit and multinomial logistic regressions of the treatment
variables on a set of relevant covariates.
Following Caliendo and Kopeinig (2008) the following variables are
included: Size of the firm, young firm dummy, in house R&D
department dummy, location of the firm’s core market, country of
origin of the firm, sector dummies.
Once the propensity scores are recovered, we use these measures to
proceed to matching and then to PSM estimations.
The most straightforward way to asses the quality of the matching
is through a graphic analysis of PS density distributions before and
after the matching

Propensity score distributions before and after matching
Policy Procurement Policy R&D Policy Procurement Only Policy R&D Only

OECD workshop on measuring the link between public procurement, R&D and innovation. "Demand side vs. supply side technology policy: hidden treatment and new empirical evidence on the technology policy mix".

Recommended

Recommended

More Related Content

Similar to OECD workshop on measuring the link between public procurement, R&D and innovation. "Demand side vs. supply side technology policy: hidden treatment and new empirical evidence on the technology policy mix".

Similar to OECD workshop on measuring the link between public procurement, R&D and innovation. "Demand side vs. supply side technology policy: hidden treatment and new empirical evidence on the technology policy mix". (20)

More from STIEAS

More from STIEAS (20)

Recently uploaded

Recently uploaded (20)

OECD workshop on measuring the link between public procurement, R&D and innovation. "Demand side vs. supply side technology policy: hidden treatment and new empirical evidence on the technology policy mix".