This keynote presentation discusses research impact and its measurement in the field of library and information science (LIS). The presentation makes several key points: LIS impact takes many forms; measuring research impact is challenging due to issues like attribution of outcomes; research impact matters and is increasingly being used to measure research quality; and various strategies can be used to extend the reach of research outcomes, like employing action research approaches and targeting dissemination. The presentation also discusses themes around measuring impact found in the LIS literature and provides examples of approaches that can improve research impact in practice based on preliminary findings from the RiLIES research project.

1. Project output versus influence in
practice
Impact as a dimension of research
quality
6th International Conference on Evidence based Librarianship and Information
Practice
Keynote presentation by Professor Hazel Hall, Director, Centre for Social
Informatics, 30th June 2011

2. http://lisresearch.org
http://lisresearch.org

4. If you are tweeting this
If you are tweeting this
session, please refer to me
session, please refer to me
as @hazelh. Thank you!
as @hazelh. Thank you!

5. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact measurement matters
 Where impact is measured as a dimension of
research quality, a number of strategies can be
deployed to extend project outcome reach

6. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact measurement matters
 Where impact is measured as a dimension of
research quality, a number of strategies can be
deployed to extend project outcome reach

7. Impact themes in LIS literature
Impact themes in LIS literature
1. Evaluation of services – general use (debated)
1. Evaluation of services – general use (debated)

8. Impact themes in LIS literature
Impact themes in LIS literature
2. Evaluation of service impact on end user behaviours
2. Evaluation of service impact on end user behaviours

9. Impact themes in LIS
Impact themes in LIS
literature
literature
3. Evaluation of services as
3. Evaluation of services as
related to specific
related to specific
outcomes
outcomes

10. Impact themes in LIS literature
Impact themes in LIS literature
4. How to evaluate services
4. How to evaluate services

11. Impact themes in LIS literature
Impact themes in LIS literature
5. Bibliometrics
5. Bibliometrics

12. Impact themes in LIS literature
Impact themes in LIS literature
5. Bibliometrics – publication impacts
5. Bibliometrics – publication impacts

13. Impact themes in LIS literature
Impact themes in LIS literature
6. Bibliometrics research evaluation
6. Bibliometrics research evaluation
We can stand on the shoulders of giants
We can stand on the shoulders of giants
because librarians and information
because librarians and information
scientists have maintained a curated record
scientists have maintained a curated record

14. Impact themes in LIS literature
Impact themes in LIS literature
7. LIS research output  LIS practice
7. LIS research output  LIS practice

15. Impact themes in LIS literature
Impact themes in LIS literature
7. LIS research output  LIS practice
7. LIS research output  LIS practice

16. Impact themes in LIS literature
Impact themes in LIS literature
8. LIS practice  LIS research
8. LIS practice  LIS research
(Brookes, 1976)
(Brookes, 1976)

17. Impact routes vary
Impact routes vary
Cumulative and indirect effect of
Cumulative and indirect effect of
exposure to research output
exposure to research output
Direct policy mandate that
Direct policy mandate that
derives from a particular study
derives from a particular study

18. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact matters
 Where impact is measured as a dimension of
research quality, a number of strategies can
be deployed to extend project outcome reach

19. Measurement challenge 1
Measurement challenge 1
Attribution, and cause and effect, are
Attribution, and cause and effect, are
not always accurate nor obvious
not always accurate nor obvious

20. Greer Garson
Greer Garson
Best actress 1942 for role in Mrs Miniver
Best actress 1942 for role in Mrs Miniver
Acceptance speech 5 minutes 30 seconds
Acceptance speech 5 minutes 30 seconds
Measurement challenge 1
Measurement challenge 1
Attribution, and cause and effect, are
Attribution, and cause and effect, are
not always accurate nor obvious
not always accurate nor obvious

21. Measurement challenge 2
Measurement challenge 2
Sources of mediated training material are
Sources of mediated training material are
not always appreciated by practitioners
not always appreciated by practitioners

22. Measurement challenge 3
Measurement challenge 3
End-users are not normally aware of research
End-users are not normally aware of research
effort to support services design and delivery
effort to support services design and delivery

23. Measurement challenge 4
Measurement challenge 4
Certain types of research output attract more
Certain types of research output attract more
attention than others
attention than others

24. Measurement challenge 5
Measurement challenge 5
The impact of some is overwhelming
The impact of some is overwhelming
(and disproportionate)
(and disproportionate)

25. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact matters
 Where impact is measured as a dimension of
research quality, a number of strategies can be
deployed to extend project outcome reach

26. Practitioner interest in research output
Practitioner interest in research output
Increases value of research undertaken
Increases value of research undertaken
Narrows research-practice gap 
Narrows research-practice gap 
benefits health of the profession
benefits health of the profession

27. Local contexts influence attitudes
Local contexts influence attitudes
towards impact measurement
towards impact measurement
REF2014 in the UK
REF2014 in the UK

28. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact matters
 Where impact is measured as a dimension of
research quality, a number of strategies can
be deployed to extend project outcome reach

29. NB presented here are strong
NB presented here are strong
messages that have emerged from
messages that have emerged from
RiLIES project empirical data
RiLIES project empirical data
analysed to date
analysed to date
http://lisresearch.org/rilies-project/
http://lisresearch.org/rilies-project/

30. Literature review; poll; focus
Literature review; poll; focus
groups; case studies; survey
groups; case studies; survey

31. Research ideas for impact in practice
Research ideas for impact in practice
Focus on research areas of direct relevance
Focus on research areas of direct relevance
to practitioners, both in terms of subject
to practitioners, both in terms of subject
and context -- time, place, sector
and context time, place, sector
(Perth and London focus groups)
(Perth and London focus groups)

32. Research approach for impact in practice
Research approach for impact in practice
Deploy action research approach
Deploy action research approach
Mix in development and engagement, e.g. capacity
Mix in development and engagement, e.g. capacity
building as hook, workshops throughout project
building as hook, workshops throughout project
Encourage community creation around project for
Encourage community creation around project for
sustainability
sustainability
Involve high profile, prestigious partners
Involve high profile, prestigious partners
(Evalued, Evaluating the impact of clinical librarian
(Evalued, Evaluating the impact of clinical librarian
services, Open to all case studies)
services, Open to all case studies)

33. Project scale and diversity for impact in practice
Project scale and diversity for impact in practice
Data collection needs to be on a scale for project
Data collection needs to be on a scale for project
recommendations to be applicable elsewhere
recommendations to be applicable elsewhere
Ensure that projects are outward-looking to
Ensure that projects are outward-looking to
safeguard against accusations of bias
safeguard against accusations of bias
(Perth focus group)
(Perth focus group)

34. Face-to-face dissemination for impact in practice
Face-to-face dissemination for impact in practice
Exploit face-to-face dissemination as the practitioners’
Exploit face-to-face dissemination as the practitioners’
favoured route
favoured route
Target dissemination according to career stage preferences
Target dissemination according to career stage preferences
Invest project budget in training activities for project legacy
Invest project budget in training activities for project legacy
Launch events effective for policy maker target audiences
Launch events effective for policy maker target audiences
(Poll, focus groups, case studies)
(Poll, focus groups, case studies)

35. Accessible textual sources for impact in practice
Accessible textual sources for impact in practice
Tailor a variety of reports according to intended audiences:
Tailor a variety of reports according to intended audiences:
tools they can access, career stage, sector…
tools they can access, career stage, sector…
Make reports accessible: easy to locate e.g. open access;
Make reports accessible: easy to locate e.g. open access;
understandable, e.g. possible to act on project outcomes
understandable, e.g. possible to act on project outcomes
Work with profile dissemination partners (e.g. HEFCE, JISC,
Work with profile dissemination partners (e.g. HEFCE, JISC,
OCLC, Pew, RIN, SCONUL) to enhance authority of work and
OCLC, Pew, RIN, SCONUL) to enhance authority of work and
increase its reach
increase its reach
Ensure work is picked up by digest/alert services and bloggers
Ensure work is picked up by digest/alert services and bloggers

36. http://www.twitter.com/LISResearch
http://www.twitter.com/LISResearch
provides an alert services
provides an alert services
Conference registration open
Conference registration open
Training day announcement
Training day announcement
Publication of research report
Publication of research report
Study day announcement
Study day announcement
Conference registration to close
Conference registration to close
Conference registration open
Conference registration open
Study day announcement
Study day announcement
Publication of blog post
Publication of blog post

37. Exploit network hubs
Exploit network hubs
5 key individuals in UK
5 key individuals in UK
emerge from RiLIES project…
emerge from RiLIES project…

38. 5 UK research connectors
5 UK research connectors
1. Rhona Arthur @RhonaEA
1. Rhona Arthur @RhonaEA

39. 5 UK research connectors
5 UK research connectors
2. Andrew Booth @scharrlock
2. Andrew Booth @scharrlock

40. 5 UK research connectors
5 UK research connectors
3. Phil Bradley @Philbradley
3. Phil Bradley @Philbradley

41. 5 UK research connectors
5 UK research connectors
4. Alison Brettle @BrettleAli
4. Alison Brettle @BrettleAli

42. 5 UK research
5 UK research
connectors
connectors
5. Hazel Hall @hazelh
5. Hazel Hall @hazelh

43. Face-to-face dissemination for impact in practice
Face-to-face dissemination for impact in practice
What happens when formal opportunities for face-to-
What happens when formal opportunities for face-to-
face dissemination are restricted:
face dissemination are restricted:
training/travel budgets cut?
training/travel budgets cut?
reduced professional memberships?
reduced professional memberships?
… and when research infrastructures are fractured? This
… and when research infrastructures are fractured? This
has been addressed in UK in the formation of the
has been addressed in UK in the formation of the
Library and Information Science Research Coalition
Library and Information Science Research Coalition

44. Researcher priority
Researcher priority
Present research output in an accessible
Present research output in an accessible
way
way
Service manager priority
Service manager priority
Facilitate research output consumption,
Facilitate research output consumption,
e.g. reverse mentoring, job descriptions,
e.g. reverse mentoring, job descriptions,
reporting back from conferences
reporting back from conferences
Course/accreditation priority
Course/accreditation priority
Application of research in practice
Application of research in practice

45. A greater role for social media?
A greater role for social media?
Currently divides the practitioner
Currently divides the practitioner
community, e.g. blogs very popular with
community, e.g. blogs very popular with
academic librarians but inaccessible to
academic librarians but inaccessible to
some others; Twitter essential to some,
some others; Twitter essential to some,
irrelevant to others
irrelevant to others
http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/

46. A greater role for self-organising
A greater role for self-organising
networks as communities of practice?
networks as communities of practice?
(with technical infrastructure provided by
(with technical infrastructure provided by
social media)
social media)

47. Research funding to support researcher-
Research funding to support researcher-
practitioner links
practitioner links
http://www.lisresearch.org/dream-project
http://www.lisresearch.org/dream-project

48. DREaM project launch conference
DREaM project launch conference
Tuesday 19th July, London
Tuesday 19th July, London
Places still available
Places still available

49. Key points
 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact measurement matters
 Where impact is measured as a dimension of
research quality, a number of strategies can be
deployed to extend project outcome reach
 Preliminary findings of RiLIES project dominated
by influence of dissemination strategy

50. Bringing it back to Salford:
anyone recognise these lads?

51. Bringing it back to Salford:
anyone recognise these lads?

52. What difference does it make?
(1984)

54. Playlist 20 March 1984
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time

55. Playlist 20 March 1984
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time

56. Playlist 20 March 1984
Playlist 20 March 1984
Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time

57. Photo credits
Image: 'I Can Gather All the News I+Need+on+the+Weather+Report' Image: 'Greer Garson & Robert Young'
http://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154
Image: 'These Books are Sweet' Image: ‘Waiting for the library to open’
http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set-
72157602122007409
Image: ‘Radar Graph – Library Usage v Final Grades'
http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: 'BC007'
http://www.flickr.com/photos/51548378@N05/4745027706
Image: '50lb weights'
http://www.flickr.com/photos/62327186@N00/2185667691 Image: 'Money to burn'
http://www.flickr.com/photos/8363329@N05/811315055
Image: 'Article-level metrics at PLoS'
http://www.flickr.com/photos/14829735@N00/3929622080 Image: 'Warhol's Light Bulbs'
http://www.flickr.com/photos/66814335@N00/3036254720
Image: '365: day 119'
http://www.flickr.com/photos/33563858@N00/2451800342 Image: 'Cheers'
http://www.flickr.com/photos/71477195@N00/204879104
Image: 'The locker misery'
http://www.flickr.com/photos/33685308@N00/513100970 Image: 'P1010121'
http://www.flickr.com/photos/67614763@N00/4304374413
Image: '90" engine valves'
http://www.flickr.com/photos/61132483@N00/2975215 Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog'
http://www.flickr.com/photos/49503019876@N01/1824234195
Image: 'Oscar'
http://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead)
by Stephen Wright
Image: 'ed vaizey'
http://www.flickr.com/photos/25414047@N00/4065621328

58. Reference & acknowledgements
 Brookes, B.C. (1976). A new paradigm for
information science. The Information Scientist,
10(3), 103-12.
 Peter Cruickshank, Ella Taylor-Smith, Jenny
Gebel (RiLIES project team)
 Stephanie Kenna (LIS Research Coalition)
 All those who have contributed data to date

59. Professor Hazel Hall
h.hall@napier.ac.uk @hazelh