Intervento al meeting L'Urgenza di Ripensare l'Economia - Rethinking Economics, Francesco Saraceno (Sciences-Po e autore di "La scienza inutile") Lorenzo Fioramonti (Viceministro Miur) Francesco Sylos Labini (Cnr e autore di "Rischio e previsione") Pasquale Tridico (Università Roma Tre) Gabriele Guzzi (Rethinking Economics Italia)

1. Ripensare la valutazione:
lezioni dall’ economia
Francesco Sylos Labini
Enrico Fermi Center
Institute for Complex Systems, CNR
 Roars.it

2. • Domanda della Regina
• Previsioni
• Valutazione
• Cambio di paradigma

3. • Domanda della Regina
• Previsioni
• Valutazione
• Cambio di paradigma

5. On 5 November (2008), the Queen must have embarrassed the economists at LSE
by asking her Question, which presupposed that all of them had failed to foresee
the biggest Global Financial Crisis since 1929. The Queen’s Question clearly implied
that something had gone seriously wrong with research in economics.

6. On 5 November (2008), the Queen must have embarrassed the economists at LSE
by asking her Question, which presupposed that all of them had failed to foresee
the biggest Global Financial Crisis since 1929. The Queen’s Question clearly implied
that something had gone seriously wrong with research in economics.
Yet in the results of the Research Assessment Exercise, published the next month,
research in economics was declared to be, by some margin, the best research in any
subject in the UK. Moreover research in economics at LSE was given a grade of 3.55
out of 4.
Who was right in their judgement? The Queen or the RAE committee?

7. • Domanda della Regina
• Previsioni
• Valutazione
• Cambio di paradigma

12. “Solo 2 delle 60 recessioni che si sono
verificate tra il 1988 e il 1998 sono state
previste con un anno di anticipo”
(Loungani 2001, International Jour. Of
Forecasting)

26. Weather forecast

29. untori

34. • Domanda della Regina
• Previsioni
• Valutazione
• Cambio di paradigma

35. On 5 November, the Queen must have embarrassed the economists at LSE by asking
her Question, which presupposed that all of them had failed to foresee the biggest
Global Financial Crisis since 1929. The Queen’s Question clearly implied that
something had gone seriously wrong with research in economics.
Yet in the results of the Research Assessment Exercise, published the next month,
research in economics was declared to be, by some margin, the best research in any
subject in the UK. Moreover research in economics at LSE was given a grade of 3.55
out of 4. Who was right in their judgement?

36. •Research Assessment Systems began with the Research Assessment Exercise
(or RAE) which was introduced into the UK by Thatcher in 1986 and continued by
Blair.
• At intervals of a few years, RAEs are carried out in all the universities of the UK.
• The first step is to appoint a committee of assessors in each subject. These
assessors are usually academics working in the field in question in the UK. Next
most members of each department in a subject have to select a set of pieces of
their research.
• The department then submits all these pieces of research produced by its
members to the assessment committee. The members of the committee study this
research output, and, on its basis, grade the department on a scale running from
very good downwards.
RAE/REF

37. Kuhn in his (1962) The Structure of
Scientific Revolutions presents a
view of the natural sciences which
has become very well-known and
quite widely accepted.

38. •According to Kuhn, mature natural sciences develop
for the most part in a manner which he describes as
‘normal science’. During a period of normal science,
all the scientists working in the field accept the same
framework of assumptions which Kuhn calls a
‘paradigm’.

39. •However, these periods of normal science are, from
time to time, interrupted by scientific revolutions in
which the dominant paradigm of the field is
overthrown and replaced by a new paradigm.

40. •The difference between the natural sciences and
the social sciences can be put as follows. In the
natural sciences, outside revolutionary periods, all
the scientists accept the same paradigm.
•In the social sciences, however, social scientists
are divided into competing schools. Each
school has its own paradigm, but these paradigms
are often very different from each other.

41. •The contrast is between a single paradigm and a
multi-paradigm situation.

42. •As far as the natural sciences are concerned, we can
illustrate this with the example of theoretical physics. In this
field, all scientists accept a paradigm whose core consists of
relativity theory and quantum mechanics.

43. •However, they would argue, relativity and quantum
mechanics work very well, and so it is sensible to accept
them for the time being. If we turn now to economics we find
a very different situation

44. The economics community is divided into different schools.
The members of each of these schools may indeed share a
paradigm, but the paradigm of one school can be very
different from that of another. Moreover the members of one
school are often extremely critical of the views of members
of another school. The school of economics which has the
most adherents at present is neoclassical economics.

45. These different schools are associated with different political
ideologies: Neo-Classical Economics, the various versions
of Keynesianism, and Marxist Economics. These schools
are arranged on a political spectrum running from the right to
the left.

46. •Our examination of the community of researchers in economics
has led us to the following picture. This community is divided into
a number of different schools of thought A, B, C, …, each with its
own paradigm.

47. •Our examination of the community of researchers in economics
has led us to the following picture. This community is divided into
a number of different schools of thought A, B, C, …, each with its
own paradigm.
•The members of each school have a very low opinion of the
research work produced by the other schools.

48. •Our examination of the community of researchers in economics
has led us to the following picture. This community is divided into
a number of different schools of thought A, B, C, …, each with its
own paradigm.
•The members of each school have a very low opinion of the
research work produced by the other schools.
•Now if a Research Assessment System (or RAS) is applied to
such a community, what result will it give?

49. •The research work of the members of whichever school has the
largest number of members will receive the highest valuation. So if
school A is in the majority, the members of school A will receive the
highest valuation. If school B is in the majority, then the members of
school B will receive the highest valuation, and so on
.
•The valuation received by the members of a
particular school X will be roughly proportional
to the number of members of that school

51. • I have argued that the effect of a research assessment system on economics is to drive the
economics community away from pluralism and towards monism.
• I have also argued that a pluralistic economics community is much more likely to produce
good research than a monistic one.
• Putting these two claims together, the conclusion is that a research assessment will
make research in economics worse rather than better. Hence there is a strong case for
abolishing research assessment systems in economics where they exist, and not
introducing any new ones.

53. Competition
Quality

56. • Domanda della Regina
• Previsioni
• Valutazione
• Cambio di paradigma

57. •Forma di controllo sulla scienza da parte di
organismi di governo: opportunità politica del
lavoro dello scienziato
•Distribuzione delle risorse: Finanziamento
delle ricerce più “utili” alla società e di
maggiore qualità
Valutazione, politica e ricerca

58. Se una ipotetica commissione che avesse dovuto
decidere del finanziamento di un “progetto” al tempo
T0 che poi ha portato al “Nobel” al tempo T1>T0, ed
avesse applicato un dato criterio (ad esempio
bilbiometrico) avrebbe preso la decisione giusta?
58
Valutare la valutazione: come?

59. 59

60. 60
Analisi dei criteri di valutazione
da un punto di vista storico

61. 61
Strategia 1: divisione della torta sulle eccellenze

62. Excellence and reality
Rewarding what is today recognised as
excellence is trivial !

63. “Reinforce excellence, dynamism and
creativity of European research"

64. “ERC funds excellent researchers”
ERC
Nobel Prize

65. 65
Strategia 1: divisione della torta sulle eccellenze
Strategia 2: spazio alle fluttuazioni

66. • Ricercatori (quasi) outsiders
• Laboratorio/università marginale
• Sorpresa rispetto stato dell’arte
Qualche scoperta

67. Superconduttività ad alta temperatura
Premio Nobel in Fisica 1987

68. Superconduttività ad alta temperatura
Premio Nobel in Fisica 1987

69. Superconduttività ad alta temperatura
Premio Nobel in Fisica 1987

70. Microscopio a effetto Tunnel
Premio Nobel per la Fisica 1987

71. Microscopio a effetto Tunnel
Premio Nobel per la Fisica 1987

72. Magneto Resistenza Gigante
Premio Nobel in Fisica 2007

73. Magneto Resistenza Gigante

74. Magneto Resistenza Gigante

75. Scanning Tunneling Microscope

76. Magneto Resistenza Gigante

77. Grafene
Premio Nobel in fisica 2010

78. 2010
Grafene

79. 2010
Grafene

81. 2010
Graphene

82. Rischio e Ricerca

83. Rischio e Ricerca

85. Research is risky !
The real problem is to understand whom to reward
today, among the large magma of good quality
researchers, and how to pick those who would become
excellent tomorrow !

86. “La storia della scienza è stata e deve
essere una storia di programmi di ricerca
in competizione … prima comincia la
competizione e meglio è per il progresso”.
(1970, Imre Lakatos)
Competizione delle idee vs competizione
ricercatori

87. We conclude that scientific impact (as reflected by
publications) is only weakly limited by funding. We
suggest that funding strategies that target
diversity, rather than “excellence”, are likely to
prove to be more productive.
Research is risk

88. Research is risk

93. Risk in research and innovation requires
• Diversification
• Adaptability
• Cooperation
• Long times
How can I reach the long
term if I do not survive in the
short one?