The document discusses how citation metrics have impacted the development of economics as a discipline, focusing on how positive feedback mechanisms have led to path dependency and the dominance of neoclassical theories. It also analyzes citation metrics as a tool to examine the state of economics by looking at factors like concentration, reactivity to incentives, diversity of ideas, and disciplinary identity. The use of citation data is discussed from both a cognitive perspective to understand information flows and an evaluative perspective to assess research performance.

1. Jakob Kapeller
University Duisburg-Essen
Chair for Socioeconomics
www.jakob-kapeller.org | www.icae.at | www.heterodoxnews.com
Amiens, 7 December 2016
Citation metrics
and the development of economics

2. Jakob Kapeller
Agenda
• Citation metrics and the development of economics I:
The impact of citation metrics on the development of economics
• The nature of citations: some basic observations
• The development of economics: a path-dependent evolution
• The impact of the rise of citation metrics on the evolution of economics
• Citation metrics and the development of economics II:
Citation analysis as a tool to analyze the state & development of economics
• Concentration: The strong internal hierarchy in economics
• Reactivity: Economists and incentives
• Diversity: Practicing path-dependency
• Identity: Thinking like an economist - isolation as a source of „superiority“
2

3. Preliminaries:
citation data

4. Jakob Kapeller
On the nature of citation data…
4

5. Jakob Kapeller
“
Merton 1968, 58
[T]he Matthew effect consists in the accruing of
greater increments of recognition for particular
scientific contributions to scientists of considerable
repute and the withholding of such recognition from
scientists who have not yet made their mark.“
How do patterns of attention arise?
Theory: Positive feedback effects and power-law distributions
5

6. Jakob Kapeller
Positive feedback, Matthew-effects
and power-law distributions
6
6
10
0
10
2
10
4
word frequency
10
0
10
2
10
4
10
0
10
2
10
4
citations
10
0
10
2
10
4
10
6
10
0
10
2
10
4
web hits
10
0
10
2
10
4
10
6
10
7
books sold
1
10
100
10
0
10
2
10
4
10
6
telephone calls received
10
0
10
3
10
6
2 3 4 5 6 7
earthquake magnitude
10
2
10
3
10
4
0.01 0.1 1
crater diameter in km
10
-4
10
-2
10
0
10
2
10
2
10
3
10
4
10
5
peak intensity
10
1
10
2
10
3
10
4
1 10 100
intensity
1
10
100
10
9
10
10
net worth in US dollars
1
10
100
10
4
10
5
10
6
name frequency
10
0
10
2
10
4
10
3
10
5
10
7
population of city
10
0
10
2
10
4
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
(j) (k) (l)
FIG. 4 Cumulative distributions or “rank/frequency plots” of twelve quantities reputed to follow power laws. The distributions
were computed as described in Appendix A. Data in the shaded regions were excluded from the calculations of the exponents
in Table I. Source references for the data are given in the text. (a) Numbers of occurrences of words in the novel Moby Dick
0 50 100 150 200 250
heights of males
0
2
4
percentage
0 20 40 60 80 100
speeds of cars
0
1
2
3
t: histogram of heights in centimetres of American males. Data from the National Health Examination Survey,
(US Department of Health and Human Services). Right: histogram of speeds in miles per hour of cars on UK
Data from Transport Statistics 2003 (UK Department for Transport).
0 2×10
5
4×10
5
population of city
0
0.001
0.002
0.003
0.004
percentageofcities
10
4
10
5
10
6
10
7
10
-8
10
-7
10
-6
10
-5
10
-4
10
-3
10
-2
t: histogram of the populations of all US cities with population of 10 000 or more. Right: another histogram of the
but plotted on logarithmic scales. The approximate straight-line form of the histogram in the right panel implies
stribution follows a power law. Data from the 2000 US Census.
is determined by the requirement that the
n p(x) sum to 1; see Section III.A.)
Power-law distributions are the subject of this arti-
cle. In the following sections, I discuss ways of detecting
Newman (2006): Power Laws, Pareto Distributions
and Zipf‘s Law.

7. Path-dependency in economic thought

8. Jakob Kapeller
„Why is economics not an evolutionary science?“
Path-dependency in economics
• Positive feedback effects are also at work in
economics, due to its strong paradigmatic priors.
• Positive feedback effects may take different
specific forms (direct, indirect, learning,
expectational…).
• Positive feedback drives path-dependency:
past success makes future success easier
(technological standards, popularity of websites
or actors, wealth, academic prestige/
reputation…)
• Path-dependency: Reproduction and
stabilization of some technological/social/
organizational/paradigmatic standard due to
positive feedback effects.
8
JOURNAL OF ECONOMIC ISSUES
Vol. XLIII No. 4 December 2009
DOI 10.2753/JEI0021-3624430403
“Why is Economics not an Evolutionary Science?”
New Answers to Veblen’s Old Question
Leonhard Dobusch and Jakob Kapeller
Abstract: Addressing the question why Economics as a discipline is not subject to
evolutive processes itself, we explain the paradigmatical dominance of neoclassical
theories in Economics as a path dependent process. Recognizing economics as
“locked into neoclassical thinking,” we first identify three positive feedback
mechanisms leading to strong barriers to paradigmatic change: coordination,
complementarity, and learning mechanisms. In a second step, we show how actors
strategically enforce these mechanisms via distinct “amplifiers.” We then try to use
this theoretical perspective to cursorily describe potentially path-breaking strategies.
Keywords: path dependency, paradigmatic dominance, pluralism
JEL Classification Codes: A11, B20, B52
About 110 years ago, when Thorstein Veblen first asked the question “Why is
economics not an evolutionary science?” he characterized the economists of his age as
“being still content to occupy themselves with repairing a structure and doctrines and
maxims resting on natural rights, utilitarism and administrative expediency,” to
describe the traditional, axiomatic orientation of economic research (Veblen 1898,
347).
The main focus of his criticism is hereby twofold: On the one hand, he labels
the classical economic theory as purely static and therefore unable to model the
phenomenon of societal or economic change in an adequate way. Moreover, he

9. Jakob Kapeller
„Why is economics not an evolutionary science?“
Positive feedback and path-dependency in economics
• Commitment to neoclassical economics as a competitive advantage:
direct network effects due to being part of an in-group
• Prospects for getting published: Segregation of journals along paradigmatic boundaries.
• Prospects for getting a job: positive feedback effects on the supply side (education!) as well as the
demand side (hiring) of the job market in economics.
• Prospects for training the next generation: lack of funds and options in doctoral studies - many
non-mainstreamers are self-educated.
• Teaching efforts: non-mainstreamers are required to teach more (Germany, Australia), to be more
versatile when teaching and often cannot use textbooks: the economist’s prime ‚labor-saving device‘.
• Prospects for getting a grant: non-mainstream economics better submit to funding agencies
related to other disciplines (from sociology to physics).
• Outcome: Forces of selection and self-selection drive the discipline towards greater
homogeneity — see, for instance, the views of younger cohorts in Econ.
9

10. Jakob Kapeller
“
Paul Davidson’s personal recollection of
one reviewer of a failed NSF grant in
1980; cited according to King (2003, 134)
It is true that Davidson has a very good track record
and surprisingly good publications, but he marches to a
different drummer. If he is marching to a different
drummer, if his music is different, then he ought to get
his own money and not use ours.“
The subtle ways of positive feedback
10

11. Jakob Kapeller
Path-dependent paradigms
Positive feedback and path-dependency in academia
11
JOHN D. STERUN AND JASON WITTENBERG Path Dependence
Figure 11
SoJved PUZZIM
ofi
Some Positive Feedback Loops that Create Path-
Dependent Behavior
Solved Puzzle9 -
of I Relative to
Competitors
Soivod Puzzles
of Compalitors
Confidence
lr>l
ConfldencB
of CompedtOTB
Net
RBcruHment
to Competitors
previously weaker riv
clude factors related t
ment of an important
include events wholly
ness of the candidate
that disrupt the work o
tive loops confer grea
the contending theor
events can overcome
diminishes, until the
to a solution. Once su
nance of the winning
sis). Yet which partic
be a matter of chance
in the emergence pha
The prevalence of
dynamics means that
the creation and early
dition their fate. Whil
a new theory is intrins
aus: Sterman/Wittenberg (1999, 333): Path
Dependence, Comeptition, and Success in the
Dynamics of Scientific Revolution.
Organization Science.

12. Jakob Kapeller
“
(Kuhn 1962, 18)
„[N]ote briefly how the emergence of a paradigm
affects the structure of the group that practices the
field. When, in the development of a natural science,
an individual or group first produces a synthesis able to
attract most of the next generation’s practitioners, the
older schools gradually disappear.“
Path-dependent paradigms
Positive feedback and path-dependency in academia
12

13. Jakob Kapeller
Path-dependent paradigms
Positive feedback and path-dependency in academia
• Direct network effect: If some standard xi is adopted by some user aj, then xi
becomes more attractive for other users.
• Matthew-effect: If some researcher xi is cited by some contribution aj, then xi
becomes more attractive for future references.
13

14. Jakob Kapeller
Path-dependent paradigms
Positive feedback and path-dependency in academia
• Indirect network effect: If users want to use some feature y, and if y is inherently
dependent on some standard xi, then xi becomes more attractive for those users,
who want to use y.
• Applied to citation metrics (in econ): If researchers want to use citation metrics for
evaluation and distributing prestige (y), and if y indicates that approach pk and
researcher xi are reputable, then approach pk/approach xi will become more
attractive in the future.
14

15. Jakob Kapeller
Path-dependent paradigms
Illustration and data
15
Average number of
annual submissions
before inclusion into the
JCR
Average number of
annual submissions after
inclusion into the JCR
Average growth in
annual submissions after
inclusion into the JCR
87 Submissions/Year 156 Submissions/Year 78.8%
Table 1: Aggregated submission data from three economic journals joining the JCR in 2010/11.
While Table 1 gives a first intuition on the impact of the indirect network effect
introduced by evaluative scientometrics, it focuses on special cases – newcomers, so
to say, for whom the inclusion in the JCR obvious implies a boost in attractiveness.

16. Applications:
Two perspectives on citation data

17. Jakob Kapeller
“
Minger/Leydesdorff 2015
Scientometrics is the study of the quantitative aspects
of the process of science as a communication system.“
Scientometrics
17

18. Jakob Kapeller
Why „two perspectives“?
• Scientometric data are used either positively or normatively...
• ...to analyze the properties of scientific conversation as well the development of
scientific fields: cognitive scientometrics
• ...to evaluate scientific „performance“ on various levels: evaluative
scientometrics
• Decisive Differences: What is a „group of highly talented geniuses“ from one
perspective, might look like a „citation cartel“ from the other.
18

19. Empirics:
concentration

20. Jakob Kapeller
Concentration: Economists focus on Economics
20
94 Journal of Economic Perspectives
(against 60 times in the ASR), while Gary Becker was reaping 41 citations in the ASR
(106 in AER). During the same period Max Weber and Mark Granovetter received
four mentions each in the AER, but James Heckman was cited 25 times in the ASR
by sociologists, and Oliver Williamson, 13.3
From the vantage point of sociologists, geographers, historians, political scien-
tists, or even psychologists, economists often resemble colonists settling on their
Table 1
Citations from the Flagship Journal to Articles Published in the 25 Top Journals in
Each Discipline, 2000–2009
(as a percentage of total citations in each journal)
Cited journals (% of all references)
Total number of
papers/citations
from this journalCiting journal
Top 25
economics
journals
Top 25
political science
journals
Top 25
sociology
journals
American Economic Review 40.3% 0.8% 0.3% 907/
29,958
American Political Science Review 4.1% 17.5% 1.0% 353/
19,936
American Sociological Review 2.3% 2.0% 22% 399/
23,993
Source: Compiled by the authors from the electronic Institute for Scientific Information’s Web of Social
Science. The high number of papers and cites in the AER is due to the Papers and Proceedings. We also
looked at this data without the Papers and Proceedings. The patterns are not significantly different.
Fourcade et al. (2015):
The superiority of
economists. JEP.
• cognitive scientometrics: economic discourse is more self-contained and less open for
interdisciplinary research.
• evaluative scientometrics: economic research is superior - it gets cited by others, but has no need
for external input itself.

21. Jakob Kapeller
Concentration:
Economists‘ „Top Five“ - a „good-old-boys network“?
21

22. Jakob Kapeller
Concentration: Interdisciplinary comparisons
22

23. Jakob Kapeller
Two interpretations
• evaluative scientometrics: economics has a small set of highly regarded
journals, which are able to select the papers with the highest merits. Hence,
high-quality research is easy to locate.
• cognitive scientometrics: economic discourse at the very top of the discipline
is rather self-contained - just as if top journals of economics exacerbate the
same form of group-think that pervades the discipline as a whole.
23

24. Empirics:
reactivity

25. Jakob Kapeller
“
(Espeland/Saunders 2007, 7)
Casting reactivity as a threat to validity of measures
draws attention to the dynamic nature of quantitative
authority and its capacity to intervene, sometimes
dramatically, with other forms of authority“
Reactivity and the authority of numbers
25

26. Jakob Kapeller
Reactivity in numbers?
• A simple hypothesis on reactivity
• The introduction of journal rankings redistributes prestige and attention from single
authors and contributions to journals.
• As a consequence self-reinforcing effects related to attention in academia (Merton‘s
Matthew Effects), should affect journals more strongly after the introduction of citation
metrics.
• Which change in citation patterns do we expect?
June 2008 Citation Statistics IMU ICIAM IMS
The Transactions of the AMS publishes longer articles that are usually more substantial, both in scope
and content. Over the same period of time, the Transactions published 1,165 papers (more than 25,000
pages), with citation counts ranging from 0 to 12. The average number of citations per article was .846—
about twice that of the Proceedings.
Now consider two mathematicians, one publishing a paper in the Proceedings and the other a paper in
the Transactions. Using some of the institutional practices cited above, the second would be judged
0%
10%
20%
30%
40%
50%
60%
70%
80%
0 1 2 3 4 5 6 7 8 9 10 11 12
Percentofarticles
Citations
Proceedings of the AMS
0%
10%
20%
30%
40%
50%
60%
0 1 2 3 4 5 6 7 8 9 10 11 12
Percentofarticles
Citations
Transactions of AMS
Adler, Ewing, Taylor (2008. 11): Citation Statistics.
URL: http://www.mathunion.org/fileadmin/IMU/
Report/CitationStatistics.pdf.

27. Jakob Kapeller
Reactivity in numbers?
27

28. Jakob Kapeller
Reactivity in numbers?
28
• evaluative scientometrics: Average quality
of papers in top-journals is rising faster,
than the quality of the top-papers.
• cognitive scientometrics: academic
prestige is transferred to journals, which
makes Matthew-Effects for authors and
articles less pronounced (at least within a
top journal).

29. Empirics:
diversity

30. Jakob Kapeller
Economics as a „contested discipline“
30
Dobusch/Kapeller (2012): Heterodox United vs.
Mainstream City? Sketching a framework for
interested pluralism in economics. Journal of Economic
Issues.

31. Jakob Kapeller
Diversity and interparadigmatic Competition
• Empirical Question: How do mainstream and heterodox economists
interact?
• Based on 1989-2008 data from Web of Science related to all the interactions between 13
„top“ mainstream and 13 „top“ heterodox journals.
31

32. Jakob Kapeller
Diversity and interparadigmatic Competition
32
• cognitive scientometrics: heterodox economics is „open“ or „pluralist“, mainstream economics is
„closed“.
• evaluative scientometrics: Mainstream economics is clearly superior.
97.15%'
47.58%'
2.85%'
52.42%'
0.00%'
20.00%'
40.00%'
60.00%'
80.00%'
100.00%'
120.00%'
in'top'13'orthodox' in'top'13'heterodox'
Interparadigma,c.Discourse.in.Economics.(198982008).
percentage'of'cita<ons'from'top'13'orthodox'journals' percentage'of'cita<ons'from'top'13'heterodox'journals'

33. Jakob Kapeller
Is it only the bias of hierarchy?
Adding a control group
33
• cognitive scientometrics: mainstream economics seems ignorant about what its paradigmatic
competitors do.
• evaluative scientometrics: Mainstream economics is clearly superior.
97.15%'
91.09%'
2.85%'
8.91%'
0.00%'
20.00%'
40.00%'
60.00%'
80.00%'
100.00%'
120.00%'
top'13'orthodox'and'top13'heterodox' top'13'orthodox'and'control'group'
Interparadigma,c.Discourse.in.Economics.(198982008):.
Adding.a.control.group.
percentage'of'cita>ons'from'top'13'orthodox'journals' percentage'of'cita>ons'from'other'group'

34. Jakob Kapeller
Diversity and interparadigmatic Competition:
After the crisis
34
• cognitive scientometrics: heterodox economics is decreasing its openness during the crisis.
• evaluative scientometrics: Mainstream economics does not require alternative viewpoints when
confronting the crisis.
97.61%'
24.58%'
2.39%'
74.42%'
0.00%'
20.00%'
40.00%'
60.00%'
80.00%'
100.00%'
120.00%'
in'top'13'orthodox' in'top'13'heterodox'
Interparadigma,c.Discourse.in.Economics.(200982013).
percentage'of'cita<ons'from'top'13'orthodox'journals' percentage'of'cita<ons'from'top'13'heterodox'journals'

35. Jakob Kapeller
Diversity and interparadigmatic Competition:
A large-scale sample
35
98.30%
69.08%
1.70%
30.92%
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
orthodox journals (> 350.000 papers} heterodox journals (> 50.000 papers)
Interparadigma,c Discourse in Economics (1969-2013):
Analyzing a large-scale sample
percentage of citaDons from orthodox journals (> 2.1M citaDons)
percentage of citaDons from heterodox journals (> 180.000 citaDons)

36. Jakob Kapeller
What does mainstream economics think about
heterodox economics?
36
My honest answer to that question was that they
don’t think about it. For the most part, the mainstream
is unaware of the existence of an ‘outside-the-
mainstream’ heterodoxy.”
“
(Colander 2010, 47)

37. Empirics:
identity

38. Jakob Kapeller
“
Justin Wolfers,
http://freakonomics.com/2010/09/21/
what-ive-been-thinking-about/
What is interesting to think about are the terms of
trade between economics and all these other
disciplines. We are clearly a net exporter to political
science and sociology.“
Identity and Performance
38

39. Jakob Kapeller
The Economics of Identity and
the identity of economics?
39
Identity and the Economics of
Organizations
George A. Akerlof and Rachel E. Kranton
O
n plebes’ first day at West Point, called R-Day, they strip down to their
underwear. Their hair is cut off. They are put in uniform. They then
must address an older cadet, with the proper salute and with the
statement: “Sir, New Cadet Doe reports to the cadet in the Red Sash for the first
time as ordered.” Plebes must stand and salute and repeat, and stand and salute
and repeat, until they get it exactly right, all the while being reprimanded for every
tiny mistake. In the summary of David Lipsky (2003, pp. 145–154), who spent four
years tracking a company of cadets at West Point: “On R-Day you surrender your old
self in stages.” But R-day is just the beginning of the training and personal
re-engineering that is to come, so that West Point graduates emerge four years later
as loyal officers in the U.S. Army. Lipsky shows that, despite some failure, this tough
program is remarkably successful in creating officers with the will to lead in battle.
Economists’ current picture of organizations and work incentives has no place
for the West Point program and the motivations it seeks to inculcate in recruits. In
a standard economic model, an individual’s preferences are fixed, and utility
depends only on pecuniary variables. The Army’s aim at West Point is to change
cadets’ preferences. They wish to inculcate non-economic motives in the cadets so
that they have the same goals as the U.S. Army. Alternatively stated, the goal of West
Point is to change the identity of the cadets, so they will think of themselves, above
all else, as officers in the U.S. army. They will feel bad about themselves—they will
lose utility—if they fall short of the ideals of such an officer. This change in identity
y George A. Akerlof is Koshland Professor of Economics, University of California, Berkeley,
California. Rachel E. Kranton is Professor of Economics, University of Maryland, College
Park, Maryland.
Journal of Economic Perspectives—Volume 19, Number 1—Winter 2005—Pages 9–32

40. Jakob Kapeller
“
Fabio Rojas,
https://orgtheory.wordpress.com/
2010/09/23/does-justin-wolfers-actually-
read-sociology/
Similarly, Akerlof is getting high fives for the Soc 101
observation that personal identity matters. It’s like a
sociologist in 2010 claiming the power and beauty of
marginal utility.“
Performance and identity
40

41. Jakob Kapeller
Performance and identity
Why it might matter to reinvent the wheel…
• basic idea:
people have believes about themselves (an ‚identity‘) they want to conform to.
41
Social Psychology (1974) Management (1989) Economics (2000)
cites after 2000: 1663 cites
6.8% from Social Psychology
1.2% from Economics
298 cites
28.1% from Management
1.7% from Economics
651 cites
8.8% from Management
1.7% from Social Psych.

42. Jakob Kapeller
“
Justin Wolfers,
http://freakonomics.com/2010/09/21/
what-ive-been-thinking-about/
What is interesting to think about are the terms of
trade between economics and all these other
disciplines. We are clearly a net exporter to political
science and sociology.“
Identity and Performance
What are ‚net exports’ telling us?
42