The document analyzes the relationship between the sophistication of firms' innovation strategies and their ability to comprehend and accurately fill out innovation surveys. It uses data from a 2015 Russian innovation survey of over 1,300 manufacturing and ICT enterprises. Latent class analysis identifies five clusters of firms that differ in their understanding of survey concepts and precision of data provided. Regression analysis finds that factors like organizational innovation, new product innovation, and use of advanced production processes are associated with better comprehension, while abandoned innovation reduces comprehension. The study aims to understand how firms' innovation competencies impact their ability to participate meaningfully in innovation data collection.

1. Institute for Statistical Studies and Economics of
Knowledge
The value of innovation statistics – is data
indifferent to the complexity of firm strategies
Vitaliy Roud,
Leonid Gokhberg
19 September 2016
Blue Sky III – Ghent

2. 2
© National Research University Higher School of Economics,
2015
Outline
Hypothesis: link between the sophistication of innovation strategy and
the comprehension of the innovation survey questionnaire
Data and method
Strategies for collecting data
Perception of the questionnaires
Accuracy of data provision
Grading the quality of survey fill-in
Testing the link between innovation strategy and quality of data
provided

3. − Qualified innovation managers of larger enterprises
are the best to recognize the core concepts
− Informal innovators would face all sorts of difficulties
3
Hypothesis: Competences to fill in the innovation survey
questionnaire and the sophistication of innovation strategy
• Cognitive testing of innovation studies – companies are not
equal in comprehension of innovation survey concepts
• Two extrema:
• This study: to operationalise the continuum of states
between total comprehension and total misunderstanding

4. Data
4
Monitoring of Innovation Behaviour of Enterprises
• Russian branch of the European Manufacturing Survey
(Consortium of 18 research centres coordinated by
Fraunhofer ISI)
• Original methodology compliant with the Oslo Manual, EU CIS and
Russian Innovation Survey
• Executed by Higher School of Economics Institute for Statistical
Studies and Economics of Knowledge biannually since 2009:
http://issek.hse.ru/innoproc/en/
• Round 2015: ~1300 enterprises in Manufacturing and ICT
• Personal interviews with the top management, stratified representative
sample (firm size, sector)
Firm-level data on:
• Conventional indicators of innovation
• Participation in national innovation surveys

5. Methods
5
Latent class analysis:
comprehension of the
innovation survey
concepts
Latent class analysis:
accuracy of data provision
Latent class analysis:
strategies of data
collection
Stage 1: understanding the diversity
Stage 2: reducing the dimensions
Principle component
analysis: comprehension
and accuracy of data
Multiple choice regression
(mlogit): grade of quality and the
sophistication of innovation
strategy
Stage 3: testing the heterogeneity
Latent class analysis – grade of
the survey participation quality:
comprehension and accuracy of
data
Innovation Strategy
Technology level
General controls

6. Strategies of data collection and provision
6
Accounting
dept
Economic/
Financial
planning
depts
Top
management
and technical
depts
Complex:
economic
planning
and
technical
depts
All types of
depts
Cluster Size 0.424 0.2037 0.1877 0.1602 0.0244
Departments/specialists involved
Accounting 0.9998 0.011 0.0009 0.3812 0.9918
Economic and financial planning departments 0.0254 0.9993 0.0016 0.8204 0.9991
Top management 0.2117 0.2146 0.4258 0.2575 0.9407
Innovation department 0.0039 0.0346 0.1597 0.2011 0.9764
Technological and technical departments 0.0632 0.1347 0.3242 0.6167 0.9987
Marketing 0.0085 0.0001 0.0406 0.3318 0.922
HR 0.0278 0 0 0.2703 0.9616
Other 0 0.0006 0.0226 0.0181 0.0092
(Share of enterprises within the cluster involving the corresponding
departments)
Enterprise departments involved in data collection: 42% are filled in exclusively by
accounting department
0
50
100
150
200
1 2 3 4 5 6 7 8 9 10 12 15 20 23 24 25 30 37
Numberofenterprises
Employees involved in data collection

7. Perception of the Innovation Survey Concepts (1)
7
Portfolio of questionnaire perceptions: perfectly relevant vs. non-applicable for the firm
Perfect
applicability
Good
applicability
Average
applicability
Average for
core questions -
Poor for
extended
Poor
applicability
Cluster Size 0.1529 0.2529 0.3549 0.1194 0.1199
Indicators
General firm characteristics (markets, human capital, etc.) 1.0554 1.7768 2.7448 1.8903 4.1099
Innovation types: product, process, organisational, marketing 1.0304 1.9682 3.0167 3.1313 4.9992
Innovation sales 1.0115 1.9536 3.1583 3.2661 4.9887
Factors hampering innovation 1.0657 1.8789 2.9124 2.9729 4.9992
Innovation expenditure 1.0286 1.7998 3.1009 4.4538 4.9938
Results of innovation 1.0007 1.8012 3.0266 4.7134 4.9993
R&D collaboration 1.0005 1.8991 3.1578 4.5982 4.9993
Information sources 1.0459 1.8845 2.9746 4.346 4.9992
Intellectual property rights protection 1.0119 1.9162 2.887 4.2468 4.9992
Purchase and selling of technologies 1.0187 1.9536 3.0489 4.6313 4.9993
Organisational and marketing innovation 1.0033 2.0015 3.2044 4.3163 4.9993
Ecological innovation 1.0311 2.0948 3.1229 4.7682 4.9993
(average score within the cluster; 1 - perfect .. 5 - poor)
Diversity reduced to a one-dimensional scale of applicability:
Perfect; Good; Average; Average for core concepts and poor for extended
framework; Poor

8. Perception of the Innovation Survey Concepts (2)
8
Portfolio of the quality of data provided: precise and verified vs. general estimates
Perfect
accuracy
Good
accuracy
Average
accuracy
Poor accuracy
Cluster Size 0.1935 0.2537 0.309 0.2438
Indicators
General firm characteristics (markets, human capital, etc.) 1.0004 1.6399 2.5739 3.8244
Innovation types: product, process, organisational, marketing 1.0226 1.8311 2.8367 5.203
Innovation sales 1.0278 1.8207 3.1647 5.2993
Factors hampering innovation 1.021 2.2062 3.4413 4.6051
Innovation expenditure 1.0005 1.9615 2.7125 5.7329
Results of innovation 1.0923 1.9613 2.9311 5.6131
R&D collaboration 1.0005 1.7636 2.8746 5.8036
Information sources 1.0006 1.8688 3.5205 5.3329
Intellectual property rights protection 1.0006 1.9527 3.3479 5.6574
Purchase and selling of technologies 1.0086 1.9161 3.3584 5.7767
Organisational and marketing innovation 1.0029 2.0603 3.68 5.6648
Ecological innovation 1.0006 2.0853 3.7091 5.7307
(average score within the cluster; 1 - perfect .. 5 - poor)
Diversity reduced to a one-dimensional scale: Perfectly precise .. Rough estimates

9. Perception of the Innovation Survey Concepts (3)
9
Dimension reduction: joint principle component analysis of applicability and
accuracy (rotated component matrix) – 3 dimensions of diversity
Core
innovation
questions
Extended
questions Accuracy
General firm characteristics (markets, human capital, etc.) .822 .257 .125
Innovation types: product, process, organisational, marketing .688 .504 .251
Innovation sales .638 .609 .179
Factors hampering innovation .687 .598 .132
Innovation expenditure .254 .818 .333
Results of innovation .271 .817 .372
R&D collaboration .224 .825 .371
Information sources .306 .816 .253
Intellectual property rights protection .239 .818 .299
Purchase and selling of technologies .244 .815 .393
Organisational and marketing innovation .222 .816 .368
Ecological innovation .180 .812 .417
General firm characteristics (markets, human capital, etc.) .162 .162 .622
Innovation types: product, process, organisational, marketing .112 .254 .851
Innovation sales .245 .298 .800
Factors hampering innovation .602 .216 .636
Innovation expenditure .037 .276 .885
Results of innovation .021 .260 .886
R&D collaboration .016 .254 .904
Information sources .274 .296 .830
Intellectual property rights protection .221 .370 .822
Purchase and selling of technologies .209 .368 .842
Organisational and marketing innovation .217 .344 .833
Ecological innovation .213 .357 .832
Applicabilityandrelevanceof
concepts
Accuracyofthedataprovided
(perfectlyverifiedvs.general
estimation)
Component
• Understanding of core concepts
(definitions of innovation and
innovation sales)
• Understanding the extended
framework (ability to account for
expenditure on innovation
activities, etc.
• Quality of data provided

10. Portfolio of questionnaire comprehension and data
precision
10
Perfect
applicability
and perfect
accuracy
Good
applicability
and accuracy
Average
applicability,
average
accuracy
Good
applicability,
poor accuracy
Poor
applicability,
poor accuracy
Cluster Size 0.1283 0.2442 0.3473 0.0813 0.1988
Applicability
General firm characteristics (markets, human capital, etc.) 1.008 1.6853 2.7245 2.3038 3.1921
Innovation types: product, process, organisational, marketing 1.0006 1.9197 3.0288 2.7295 4.0732
Innovation sales 1.0107 2.026 3.1641 2.6653 4.0347
Factors hampering innovation 1.0156 1.8933 3.0541 1.9834 3.991
Innovation expenditure 1.0006 1.9861 3.0593 2.5826 4.8272
Results of innovation 1.0006 1.9207 3.0286 2.5429 4.8938
R&D collaboration 1.0006 2.0122 3.1561 2.3815 4.9112
Information sources 1.0006 1.981 3.1801 1.7698 4.6812
Intellectual property rights protection 1.0006 1.9325 3.0562 1.8357 4.8577
Purchase and selling of technologies 1.0006 1.97 3.1171 2.3354 4.955
Organisational and marketing innovation 1.004 2.097 3.1603 2.7315 4.7066
Ecological innovation 1.0006 2.1339 3.1202 2.8391 4.9548
Accuracy
General firm characteristics (markets, human capital, etc.) 1.0004 1.4989 2.47 3.0271 3.7262
Innovation types: product, process, organisational, marketing 1.0006 1.704 2.6064 4.9991 4.7767
Innovation sales 1.0249 1.6706 2.6115 5.2474 5.309
Factors hampering innovation 1.0289 1.8454 2.9795 4.5602 4.7728
Innovation expenditure 1.0006 1.6296 2.7644 5.4212 5.0025
Results of innovation 1.0007 1.9079 2.7896 5.1648 4.9992
R&D collaboration 1.0006 1.6172 2.7383 5.6144 5.0683
Information sources 1.0092 1.7682 2.8555 4.9991 5.484
Intellectual property rights protection 1.0007 1.6838 2.7808 5.2712 5.8546
Purchase and selling of technologies 1.0007 1.6015 2.8021 5.751 5.8462
Organisational and marketing innovation 1.0042 1.7264 3.1094 5.6506 5.7324
Ecological innovation 1.0007 1.6738 3.1852 5.6957 5.8407
(average score within the cluster; 1 - perfect .. 5 - poor)

11. Determinants of questionnaire comprehension
11
Marginal effects of the variables i
Perfect
applicability
and perfect
accuracy
Good
applicability
and
accuracy
Average
applicability
, average
accuracy
Good
applicabilit
y, poor
accuracy
Poor
applicabilit
y, poor
accuracy
Number of employees (log) 0.0370** 0.0234* -0.0297 -0.00267 -0.0281
(0.0162) (0.0121) (0.0279) (0.0237) (0.0176)
New to market product innovation 0.1304** 0.0388** 0.0566 0.0149 -0.0631
(0.0506) (0.0161) (0.0312) (0.0057) (0.0858)
New to firm product innovation -0.00416 0.0364 -0.0833 0.00228 0.0488
(0.0564) (0.0527) (0.0676) (0.0466) (0.0693)
Process innovation -0.0946 0.0625 0.0679 -0.0592 0.0233
(0.0598) (0.0626) (0.0713) (0.0439) (0.0623)
Organisational innovation 0.108** -0.103* -0.0907* -0.0432* -0.128**
(0.0532) (0.0605) (0.0526) (0.0208) (0.0616)
New marketing methods 0.0490 0.0986* -0.158* -0.00809 0.0182
(0.0533) (0.0554) (0.0818) (0.0554) (0.0646)
Ongoing innovation -0.123* -0.00512 0.00673 0.0470 0.0740
(0.0699) (0.0665) (0.0769) (0.0472) (0.0677)
Abandoned innovation -1.064*** -0.0398 0.842*** 0.189** 0.0722
(0.199) (0.174) (0.168) (0.0951) (0.151)
Technologies for automation and logistics -0.00160 -0.0291 -0.0345 0.0354 0.0298
(0.0230) (0.0305) (0.0373) (0.0338) (0.0359)
Advanced production processes 0.0806** 0.0469* 0.0225 0.0313 -0.101
(0.0238) (0.0246) (0.0389) (0.0267) (0.0821)
Optimized organisational concepts 0.000690 0.0874*** -0.0278 -0.00330 -0.0570
(0.0304) (0.0279) (0.0442) (0.0250) (0.0384)
Innovation management 0.0658*** 0.0321** 0.0370** -0.0295 -0.105***
(0.0224) (0.0164) (0.0125) (0.0274) (0.0291)
Has in-house innovation effort 1.191*** 0.629*** -0.191 -0.186* -0.185*
(0.206) (0.127) (0.168) (0.107) (0.0753)
Active at the international market 0.0213** 0.0805 0.0934 -0.0594 -0.136**
(0.0079) (0.0621) (0.0654) (0.0453) (0.0644)
Base questionnaire fill-in strategy: only accounting dept
Economic and financial planning depts 0.141* 0.0402 -0.235*** 0.0585 -0.00492
(0.0743) (0.0664) (0.0878) (0.0704) (0.0763)
Top management and technical specialists 0.0898 0.0890 -0.112 0.00837 -0.0753
(0.0588) (0.103) (0.0935) (0.0686) (0.0581)
Complex: economic planning, technical depts and
others -0.0507 0.0873 -0.348*** -0.0331 0.344***
(0.0355) (0.0790) (0.0751) (0.0499) (0.0590)
All types of departments -0.0403 -0.116* -0.464*** -0.0559 0.677***
(0.0508) (0.0633) (0.0641) (0.0394) (0.0552)
Number of employees to complete the survey -0.0747* 0.0213* 0.289*** 0.00380 -0.240***
(0.0443) (0.0158) (0.0491) (0.0355) (0.0544)
Observations 401 401 401 401 401
Standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Marginal effects after mlogit choice regression: comprehension of the innovation survey questionnaire
Firm size: larger companies
Innovation:
new-to-market innovation
but even more important:
organisational innovation
Technological level:
Advanced production technologies
but even more important:
Innovation management culture
In-house innovation effort
Activity at the international market
Companies better at providing data:
Best organisation of data collection:
• Economic and financial planning depts.
• Limited number of responsible employees

12. Conclusions
12
(1) Grading the comprehension of the
innovation survey and the accuracy of data
provision:
• understanding basic definitions
• ability to measure broader indicators
• accuracy of data provided
(2) Advanced companies are better at
collecting and delivering innovation-related
data
• New-to-market product innovators
• International markets
• Advanced technology levels
(3) But organizational and innovation
culture is of higher importance
• Organisational innovation
• Advanced organisational concepts
• Innovation management culture
Key findings
(1) Need for clear guidelines (or sections)
• Explaining basic definitions
• Recommendations on identifying complex
indicators
• Instructions to increase accuracy
(2) Overcome existing bias towards
advanced and large companies
• Modular surveys should be more friendly
enterprises notable for to less sophisticated
innovation strategies and ad-hoc innovation
management processes
(3) Survey guidelines tailored for different
enterprise units
• accountants
• technical specialists
• top management
Implications

13. 13
Thank you!
vroud@hse.ru
http://issek.hse.ru
https://foresight-journal.hse.ru/en/