1. Excellence is Bullshit:
Rethinking Research Evaluation for a Networked World
@cameronneylon
http://orcid.org/0000-0002-0068-716X

2. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter being discussed”
Harry Frankfurt

3. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter being discussed”
Wikipedia

4. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter being discussed”
Wikipedia

5. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter being discussed”
Wikipedia

6. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter being discussed”
Wikipedia

7. “[The purpose of bullshit] is to
impress the listener and the reader
with words that communicate that
[…]something is being or has been
done, words that are neither true
nor false, and so obscure the facts of
the matter that should be discussed”

8. My work is excellent…

9. This talk is excellent…

10. “[…a] commitment
to excellence and
advancement on merit”

11. “[…a] commitment
to excellence and
advancement on merit”
University College London

12. “[…a] commitment
to excellence and
advancement on merit”
University College London

13. What do we mean when we use the word…
“excellence”?

14. Pursuit of excellence:
We strive
for excellence, seeking
to apply the highest
standards to benefit…
University of Queensland

15. The university pursues
academic excellence
through the creation,
preservation…
University of San Diego

16. …to strive
for excellence and
distinction…
Curtin University

17. We prize excellence
Cape Peninsula
University of Technology

18. We demand excellence
University of Westminster

19. …with world-
class excellence
across all of its
departments
Princeton

20. As a world-leading centre of
academic excellence we aim…
University of Edinburgh

21. The university counts the diversity of
its community as […] integral to its
mission of excellence, and it is
committed to diversity and
inclusiveness in both educational…
University of Maryland

22. “[…a] commitment
to excellence and
advancement on merit…
Fairness and equality…
Diversity University College London

23. excellence in…?

24. excellence in professionally
relevant teaching
University of Westminster

25. excellence in research
and education
Indiana University

26. excellence in research,
education and knowledge
exchange
Strathclyde University

27. excellence in teaching,
research, and public
engagement
Cornell University

28. …a center of
excellence for cultural,
athletic and other events
University of Toledo

29. …internationally recognized
excellence in information
technology services and
infrastructure.
Indiana University

31. What do we mean when we use the word…
“excellence”?

32. How can we determine or measure…
“excellence”?

33. We prize
academic excellence at UC Davis.
More than 500 of our faculty have
earned prestigious [..] honors[..]
rank[ings]. UC Davis

34. Pursuing excellence means being
satisfied with no less than the
highest goals we can envision.
University of Missouri

35. Pursuing excellence involves being
informed by regional, national and
global standards, as well as our
personal expectations. University of Missouri

36. “excellence” is the holy
grail of academic life
Michèle Lamont
How Professors Think

37. excellence is produced and
defined in a multitude of sites
and by an array of actors
Michèle Lamont
How Professors Think

38. excellence is produced and
defined in a multitude of sites
and by an array of actors
Michèle Lamont
How Professors Think

39. excellence is produced and
defined in a multitude of sites
and by an array of actors
Michèle Lamont
How Professors Think

40. excellence is produced and
defined in a multitude of sites
and by an array of actors
Michèle Lamont
How Professors Think

41. “excellence” tells us nothing
about how important the science is
and everything about who decides.
Jack Stilgoe
Against Excellence

42. Using the word
excellence
is a way of avoiding
a difficult conversation

43. Using the word
excellence
is a way of saying
something is being or
has been done

44. Using the word
excellence
is a way of using words
that are neither true
nor false

45. Using the word
excellence
is a way of obscuring
the facts of the matter
that should be discussed

46. Using the word
excellence
is a way of
bullshitting ourselves

47. Using the word
excellence
is a way of avoiding
talking about values

48. Impact
Research
Economic
Cultural
Education
Health
Environment

49. Impact
Research
Economic
Cultural
Education
Health
Environment
excellence

50. Impact
Research
Economic
Cultural
Education
Health
Environment

51. Impact
Research
Economic
Cultural
Education
Health
Environment
impact

52. Impact
Research
Economic
Cultural
Education
Health
Environment
quality

53. Impact
Research
Economic
Cultural
Education
Health
Environment
innovation

54. Impact
Research
Economic
Cultural
Education
Health
Environment

55. Impact
Research
Economic
Cultural
Education
Health
Environment

56. Impact
Research
Economic
Cultural
Education
Health
Environment

57. Impact
Research
Economic
Cultural
Education
Health
Environment

58. Impact
evaluation

59. …what we may be doing […]
when we make [a] value
judgement of a […] work...
Barbara Herrnstein Smith
Contingencies of Value

60. …articulating an estimate of
how well that work will serve
implicitly defined functions…
Barbara Herrnstein Smith
Contingencies of Value

61. …for an
implicitly defined audience…
Barbara Herrnstein Smith
Contingencies of Value

62. …experiencing the work under
implicitly defined conditions…
Barbara Herrnstein Smith
Contingencies of Value

63. …the work
of evaluation is to make these
implicit factors explicit.

64. Impact
Research
Economic
Cultural
Education
Health
Environment

65. Impact
There is some good news…

66. http://www.flickr.com/photos/verzo/8020565592/ CC
BY

67. https://www.flickr.com/photos/see-­‐through-­‐the-­‐eye-­‐of-­‐g/5392290809
CC
BY

68. https://www.flickr.com/photos/jenny-­‐pics/3239638494
CC
BY

73. Citation

74. Bookmark
Citation

75. Bookmark
Citation
Social
media
conversation

76. Bookmark
Citation
Previous
co-­‐authorship
Conference
co-­‐attendance
Social
media
conversation

78. Impact
There is some bad news…

93. Impact
“excellence” becomes a performance

94. Impact
…performance of
“excellence”

95. Impact
…in the place of
excellent performance

99. “[…a] commitment
to excellence and
advancement on merit…
Fairness and equality…
Diversity University College London

101. Impact
Change the narrative…

102. Impact
“soundness”

103. Impact
“practice”

104. “social practice”
Impact

106. Impact
Answers are easy…

107. Impact
…it’s the questions that are hard

108. Impact
How do we evaluate whether we are
“achieving maximum positive social,
environmental and economic
benefit”?

109. Impact
How do we evaluate whether we are
“[d]eveloping future generations of
leaders in scholarship, research, the
learned professions, the public sector,
business and innovation”?

110. Impact
How do we hold ourselves to values of
“fairness and equality, diversity,
collegiality and community-building,
inclusiveness, openness, ethically
acceptable standards of conduct…”

111. @cameronneylon
http://orcid.org/0000-0002-0068-716X
Slides: https://t.co/vNkQ9ZMhwm