SlideShare a Scribd company logo
Project output versus influence in
practice

Impact as a dimension of research
quality

6th International Conference on Evidence based Librarianship and Information
Practice
Keynote presentation by Professor Hazel Hall, Director, Centre for Social
Informatics, 30th June 2011
http://lisresearch.org
http://lisresearch.org
If you are tweeting this
If you are tweeting this
session, please refer to me
session, please refer to me
as @hazelh. Thank you!
as @hazelh. Thank you!
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact measurement matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can be
    deployed to extend project outcome reach
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact measurement matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can be
    deployed to extend project outcome reach
Impact themes in LIS literature
Impact themes in LIS literature
1. Evaluation of services – general use (debated)
1. Evaluation of services – general use (debated)
Impact themes in LIS literature
Impact themes in LIS literature
2. Evaluation of service impact on end user behaviours
2. Evaluation of service impact on end user behaviours
Impact themes in LIS
Impact themes in LIS
literature
literature
3. Evaluation of services as
3. Evaluation of services as
related to specific
related to specific
outcomes
outcomes
Impact themes in LIS literature
Impact themes in LIS literature
4. How to evaluate services
4. How to evaluate services
Impact themes in LIS literature
Impact themes in LIS literature
5. Bibliometrics
5. Bibliometrics
Impact themes in LIS literature
Impact themes in LIS literature
5. Bibliometrics – publication impacts
5. Bibliometrics – publication impacts
Impact themes in LIS literature
Impact themes in LIS literature
6. Bibliometrics research evaluation
6. Bibliometrics research evaluation
We can stand on the shoulders of giants
We can stand on the shoulders of giants
because librarians and information
because librarians and information
scientists have maintained a curated record
scientists have maintained a curated record
Impact themes in LIS literature
Impact themes in LIS literature
7. LIS research output  LIS practice
7. LIS research output  LIS practice
Impact themes in LIS literature
Impact themes in LIS literature
7. LIS research output  LIS practice
7. LIS research output  LIS practice
Impact themes in LIS literature
Impact themes in LIS literature
8. LIS practice  LIS research
8. LIS practice  LIS research
(Brookes, 1976)
(Brookes, 1976)
Impact routes vary
Impact routes vary

Cumulative and indirect effect of
Cumulative and indirect effect of
exposure to research output
exposure to research output

Direct policy mandate that
Direct policy mandate that
derives from a particular study
derives from a particular study
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can
    be deployed to extend project outcome reach
Measurement challenge 1
Measurement challenge 1
Attribution, and cause and effect, are
Attribution, and cause and effect, are
not always accurate nor obvious
not always accurate nor obvious
Greer Garson
                                   Greer Garson
                                   Best actress 1942 for role in Mrs Miniver
                                   Best actress 1942 for role in Mrs Miniver
                                   Acceptance speech 5 minutes 30 seconds
                                   Acceptance speech 5 minutes 30 seconds




Measurement challenge 1
Measurement challenge 1
Attribution, and cause and effect, are
Attribution, and cause and effect, are
not always accurate nor obvious
not always accurate nor obvious
Measurement challenge 2
Measurement challenge 2
Sources of mediated training material are
Sources of mediated training material are
not always appreciated by practitioners
not always appreciated by practitioners
Measurement challenge 3
Measurement challenge 3
End-users are not normally aware of research
End-users are not normally aware of research
effort to support services design and delivery
effort to support services design and delivery
Measurement challenge 4
Measurement challenge 4
Certain types of research output attract more
Certain types of research output attract more
attention than others
attention than others
Measurement challenge 5
Measurement challenge 5
The impact of some is overwhelming
The impact of some is overwhelming
(and disproportionate)
(and disproportionate)
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can be
    deployed to extend project outcome reach
Practitioner interest in research output
Practitioner interest in research output
Increases value of research undertaken
Increases value of research undertaken
Narrows research-practice gap 
Narrows research-practice gap 
benefits health of the profession
benefits health of the profession
Local contexts influence attitudes
Local contexts influence attitudes
towards impact measurement
towards impact measurement
REF2014 in the UK
REF2014 in the UK
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can
    be deployed to extend project outcome reach
NB presented here are strong
NB presented here are strong
messages that have emerged from
messages that have emerged from
RiLIES project empirical data
RiLIES project empirical data
analysed to date
analysed to date

http://lisresearch.org/rilies-project/
http://lisresearch.org/rilies-project/
Literature review; poll; focus
Literature review; poll; focus
groups; case studies; survey
groups; case studies; survey
Research ideas for impact in practice
Research ideas for impact in practice

Focus on research areas of direct relevance
Focus on research areas of direct relevance
to practitioners, both in terms of subject
to practitioners, both in terms of subject
and context -- time, place, sector
and context time, place, sector

(Perth and London focus groups)
(Perth and London focus groups)
Research approach for impact in practice
Research approach for impact in practice

Deploy action research approach
Deploy action research approach

Mix in development and engagement, e.g. capacity
Mix in development and engagement, e.g. capacity
building as hook, workshops throughout project
building as hook, workshops throughout project

Encourage community creation around project for
Encourage community creation around project for
sustainability
sustainability

Involve high profile, prestigious partners
Involve high profile, prestigious partners

(Evalued, Evaluating the impact of clinical librarian
(Evalued, Evaluating the impact of clinical librarian
services, Open to all case studies)
services, Open to all case studies)
Project scale and diversity for impact in practice
Project scale and diversity for impact in practice

Data collection needs to be on a scale for project
Data collection needs to be on a scale for project
recommendations to be applicable elsewhere
recommendations to be applicable elsewhere

Ensure that projects are outward-looking to
Ensure that projects are outward-looking to
safeguard against accusations of bias
safeguard against accusations of bias

(Perth focus group)
(Perth focus group)
Face-to-face dissemination for impact in practice
Face-to-face dissemination for impact in practice

Exploit face-to-face dissemination as the practitioners’
Exploit face-to-face dissemination as the practitioners’
favoured route
favoured route

Target dissemination according to career stage preferences
Target dissemination according to career stage preferences

Invest project budget in training activities for project legacy
Invest project budget in training activities for project legacy

Launch events effective for policy maker target audiences
Launch events effective for policy maker target audiences

(Poll, focus groups, case studies)
(Poll, focus groups, case studies)
Accessible textual sources for impact in practice
Accessible textual sources for impact in practice

Tailor a variety of reports according to intended audiences:
Tailor a variety of reports according to intended audiences:
tools they can access, career stage, sector…
tools they can access, career stage, sector…

Make reports accessible: easy to locate e.g. open access;
Make reports accessible: easy to locate e.g. open access;
understandable, e.g. possible to act on project outcomes
understandable, e.g. possible to act on project outcomes

Work with profile dissemination partners (e.g. HEFCE, JISC,
 Work with profile dissemination partners (e.g. HEFCE, JISC,
OCLC, Pew, RIN, SCONUL) to enhance authority of work and
 OCLC, Pew, RIN, SCONUL) to enhance authority of work and
increase its reach
 increase its reach

Ensure work is picked up by digest/alert services and bloggers
Ensure work is picked up by digest/alert services and bloggers
http://www.twitter.com/LISResearch
http://www.twitter.com/LISResearch
provides an alert services
provides an alert services

                                     Conference registration open
                                     Conference registration open

                                     Training day announcement
                                     Training day announcement


                                     Publication of research report
                                     Publication of research report

                                     Study day announcement
                                     Study day announcement

                                     Conference registration to close
                                     Conference registration to close

                                     Conference registration open
                                     Conference registration open

                                     Study day announcement
                                     Study day announcement

                                     Publication of blog post
                                     Publication of blog post
Exploit network hubs
Exploit network hubs
5 key individuals in UK
5 key individuals in UK
emerge from RiLIES project…
emerge from RiLIES project…
5 UK research connectors
5 UK research connectors
1. Rhona Arthur @RhonaEA
1. Rhona Arthur @RhonaEA
5 UK research connectors
5 UK research connectors
2. Andrew Booth @scharrlock
2. Andrew Booth @scharrlock
5 UK research connectors
5 UK research connectors
3. Phil Bradley @Philbradley
3. Phil Bradley @Philbradley
5 UK research connectors
5 UK research connectors
4. Alison Brettle @BrettleAli
4. Alison Brettle @BrettleAli
5 UK research
5 UK research
connectors
connectors
5. Hazel Hall @hazelh
5. Hazel Hall @hazelh
Face-to-face dissemination for impact in practice
Face-to-face dissemination for impact in practice

What happens when formal opportunities for face-to-
What happens when formal opportunities for face-to-
face dissemination are restricted:
face dissemination are restricted:

         training/travel budgets cut?
         training/travel budgets cut?

         reduced professional memberships?
         reduced professional memberships?

… and when research infrastructures are fractured? This
… and when research infrastructures are fractured? This
has been addressed in UK in the formation of the
has been addressed in UK in the formation of the
Library and Information Science Research Coalition
Library and Information Science Research Coalition
Researcher priority
Researcher priority
Present research output in an accessible
Present research output in an accessible
way
way

Service manager priority
Service manager priority
Facilitate research output consumption,
Facilitate research output consumption,
e.g. reverse mentoring, job descriptions,
e.g. reverse mentoring, job descriptions,
reporting back from conferences
reporting back from conferences

Course/accreditation priority
Course/accreditation priority
Application of research in practice
Application of research in practice
A greater role for social media?
                                                             A greater role for social media?

                                                             Currently divides the practitioner
                                                              Currently divides the practitioner
                                                             community, e.g. blogs very popular with
                                                              community, e.g. blogs very popular with
                                                             academic librarians but inaccessible to
                                                              academic librarians but inaccessible to
                                                             some others; Twitter essential to some,
                                                              some others; Twitter essential to some,
                                                             irrelevant to others
                                                              irrelevant to others




http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/
A greater role for self-organising
A greater role for self-organising
networks as communities of practice?
networks as communities of practice?

(with technical infrastructure provided by
(with technical infrastructure provided by
social media)
social media)
Research funding to support researcher-
Research funding to support researcher-
practitioner links
practitioner links

http://www.lisresearch.org/dream-project
http://www.lisresearch.org/dream-project
DREaM project launch conference
DREaM project launch conference
Tuesday 19th July, London
Tuesday 19th July, London

Places still available
Places still available
Key points


   LIS impact takes a number of forms
   Research impact measurement is tricky
   Research impact measurement matters
   Where impact is measured as a dimension of
    research quality, a number of strategies can be
    deployed to extend project outcome reach
     Preliminary findings of RiLIES project dominated
      by influence of dissemination strategy
Bringing it back to Salford:
anyone recognise these lads?
Bringing it back to Salford:
anyone recognise these lads?
What difference does it make?
            (1984)
Playlist 20 March 1984
Playlist 20 March 1984

Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time
Playlist 20 March 1984
Playlist 20 March 1984

Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time
Playlist 20 March 1984
Playlist 20 March 1984

Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time
Photo credits

Image: 'I Can Gather All the News I+Need+on+the+Weather+Report'                 Image: 'Greer Garson & Robert Young'
http://www.flickr.com/photos/51035555243@N01/2671536366                         http://www.flickr.com/photos/44186100@N00/2490143154

Image: 'These Books are Sweet'                                                  Image: ‘Waiting for the library to open’
http://www.flickr.com/photos/47823583@N03/4474869956                            http://www.flickr.com/photos/davepattern/1424246518/in/set-
                                                                                72157602122007409
Image: ‘Radar Graph – Library Usage v Final Grades'
http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/   Image: 'BC007'
                                                                                http://www.flickr.com/photos/51548378@N05/4745027706
Image: '50lb weights'
http://www.flickr.com/photos/62327186@N00/2185667691                            Image: 'Money to burn'
                                                                                http://www.flickr.com/photos/8363329@N05/811315055
Image: 'Article-level metrics at PLoS'
http://www.flickr.com/photos/14829735@N00/3929622080                            Image: 'Warhol's Light Bulbs'
                                                                                http://www.flickr.com/photos/66814335@N00/3036254720
Image: '365: day 119'
http://www.flickr.com/photos/33563858@N00/2451800342                            Image: 'Cheers'
                                                                                http://www.flickr.com/photos/71477195@N00/204879104
Image: 'The locker misery'
http://www.flickr.com/photos/33685308@N00/513100970                             Image: 'P1010121'
                                                                                http://www.flickr.com/photos/67614763@N00/4304374413
Image: '90" engine valves'
http://www.flickr.com/photos/61132483@N00/2975215                               Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog'
                                                                                http://www.flickr.com/photos/49503019876@N01/1824234195
Image: 'Oscar'
http://www.flickr.com/photos/45756179@N03/5068766020                            Smiths at Salford Lads Club (inside cover of The Queen is dead)
                                                                                by Stephen Wright
Image: 'ed vaizey'
http://www.flickr.com/photos/25414047@N00/4065621328
Reference & acknowledgements


 Brookes, B.C. (1976). A new paradigm for
  information science. The Information Scientist,
  10(3), 103-12.
 Peter Cruickshank, Ella Taylor-Smith, Jenny
  Gebel (RiLIES project team)
 Stephanie Kenna (LIS Research Coalition)
 All those who have contributed data to date
Professor Hazel Hall
h.hall@napier.ac.uk @hazelh

More Related Content

Similar to Project output versus influence in practice : impact as a dimension of research quality

SCC2011 - Evaluation: Facing the tricky questions
SCC2011 - Evaluation: Facing the tricky questionsSCC2011 - Evaluation: Facing the tricky questions
SCC2011 - Evaluation: Facing the tricky questions
British Science Association
 
1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos
Museo del Oro Colombia
 
Laura Eyre and Martin Marshall: Researchers in residence
Laura Eyre and Martin Marshall: Researchers in residence Laura Eyre and Martin Marshall: Researchers in residence
Laura Eyre and Martin Marshall: Researchers in residence
Nuffield Trust
 
Getting research into action: issues, challenges, solutions by Dr Sarah Morton
Getting research into action: issues, challenges, solutions by Dr Sarah MortonGetting research into action: issues, challenges, solutions by Dr Sarah Morton
Getting research into action: issues, challenges, solutions by Dr Sarah Morton
Hazel Hall
 
Research impact assessment workshop cara 170507 v140724
Research impact assessment workshop cara 170507 v140724Research impact assessment workshop cara 170507 v140724
Research impact assessment workshop cara 170507 v140724
KMb Unit, York University
 
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
Institute of Development Studies
 
Northwestern DBIR brown bag
Northwestern DBIR brown bagNorthwestern DBIR brown bag
Northwestern DBIR brown bag
University of Michigan
 
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
Food_Systems_Innovation
 
"Designing practitioner research for impact" Miggie Pickton, DARTS4
"Designing practitioner research for impact" Miggie Pickton, DARTS4"Designing practitioner research for impact" Miggie Pickton, DARTS4
"Designing practitioner research for impact" Miggie Pickton, DARTS4
ARLGSW
 
Studying implementation 2017
Studying implementation 2017Studying implementation 2017
Studying implementation 2017
Martha Seife
 
Evaluation amidst complexity: 8 questions evaluators should ask
Evaluation amidst complexity: 8 questions evaluators should askEvaluation amidst complexity: 8 questions evaluators should ask
Evaluation amidst complexity: 8 questions evaluators should ask
Ann Larson
 
PRISM-Evaluation-Toolkit-V1.pdf
PRISM-Evaluation-Toolkit-V1.pdfPRISM-Evaluation-Toolkit-V1.pdf
PRISM-Evaluation-Toolkit-V1.pdf
zoelfiqar
 
Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2
Brent MacKinnon
 
Research and policy processes
Research and policy processesResearch and policy processes
Research and policy processes
CommsConsult Ltd.
 
Research to policy processes
Research to policy processesResearch to policy processes
Impact support for research administrators
Impact support for research administratorsImpact support for research administrators
Impact support for research administrators
Esther De Smet
 
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
Institute of Development Studies
 
Assessing societal impact: what have we learned from the UK REF?
Assessing societal impact: what have we learned from the UK REF?Assessing societal impact: what have we learned from the UK REF?
Assessing societal impact: what have we learned from the UK REF?
ORCID, Inc
 
Linking Research to Practice
Linking Research to PracticeLinking Research to Practice
Linking Research to Practice
Olivier Serrat
 
Evaluation: a means to gain insights into and improve the ROER4D project
Evaluation: a means to gain insights into and improve the ROER4D project Evaluation: a means to gain insights into and improve the ROER4D project
Evaluation: a means to gain insights into and improve the ROER4D project
SarahG_SS
 

Similar to Project output versus influence in practice : impact as a dimension of research quality (20)

SCC2011 - Evaluation: Facing the tricky questions
SCC2011 - Evaluation: Facing the tricky questionsSCC2011 - Evaluation: Facing the tricky questions
SCC2011 - Evaluation: Facing the tricky questions
 
1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos
 
Laura Eyre and Martin Marshall: Researchers in residence
Laura Eyre and Martin Marshall: Researchers in residence Laura Eyre and Martin Marshall: Researchers in residence
Laura Eyre and Martin Marshall: Researchers in residence
 
Getting research into action: issues, challenges, solutions by Dr Sarah Morton
Getting research into action: issues, challenges, solutions by Dr Sarah MortonGetting research into action: issues, challenges, solutions by Dr Sarah Morton
Getting research into action: issues, challenges, solutions by Dr Sarah Morton
 
Research impact assessment workshop cara 170507 v140724
Research impact assessment workshop cara 170507 v140724Research impact assessment workshop cara 170507 v140724
Research impact assessment workshop cara 170507 v140724
 
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
 
Northwestern DBIR brown bag
Northwestern DBIR brown bagNorthwestern DBIR brown bag
Northwestern DBIR brown bag
 
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
A flexible, multi-purpose monitoring, evaluation, and learning system to supp...
 
"Designing practitioner research for impact" Miggie Pickton, DARTS4
"Designing practitioner research for impact" Miggie Pickton, DARTS4"Designing practitioner research for impact" Miggie Pickton, DARTS4
"Designing practitioner research for impact" Miggie Pickton, DARTS4
 
Studying implementation 2017
Studying implementation 2017Studying implementation 2017
Studying implementation 2017
 
Evaluation amidst complexity: 8 questions evaluators should ask
Evaluation amidst complexity: 8 questions evaluators should askEvaluation amidst complexity: 8 questions evaluators should ask
Evaluation amidst complexity: 8 questions evaluators should ask
 
PRISM-Evaluation-Toolkit-V1.pdf
PRISM-Evaluation-Toolkit-V1.pdfPRISM-Evaluation-Toolkit-V1.pdf
PRISM-Evaluation-Toolkit-V1.pdf
 
Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2Street Jibe Evaluation Workshop 2
Street Jibe Evaluation Workshop 2
 
Research and policy processes
Research and policy processesResearch and policy processes
Research and policy processes
 
Research to policy processes
Research to policy processesResearch to policy processes
Research to policy processes
 
Impact support for research administrators
Impact support for research administratorsImpact support for research administrators
Impact support for research administrators
 
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
 
Assessing societal impact: what have we learned from the UK REF?
Assessing societal impact: what have we learned from the UK REF?Assessing societal impact: what have we learned from the UK REF?
Assessing societal impact: what have we learned from the UK REF?
 
Linking Research to Practice
Linking Research to PracticeLinking Research to Practice
Linking Research to Practice
 
Evaluation: a means to gain insights into and improve the ROER4D project
Evaluation: a means to gain insights into and improve the ROER4D project Evaluation: a means to gain insights into and improve the ROER4D project
Evaluation: a means to gain insights into and improve the ROER4D project
 

More from Hazel Hall

Preparation of the PhD thesis for examination
Preparation of the PhD thesis for examinationPreparation of the PhD thesis for examination
Preparation of the PhD thesis for examination
Hazel Hall
 
RIVAL 2019-21: Did we get it right?
RIVAL 2019-21: Did we get it right? RIVAL 2019-21: Did we get it right?
RIVAL 2019-21: Did we get it right?
Hazel Hall
 
Platform to Platform project lightening talk
Platform to Platform project lightening talkPlatform to Platform project lightening talk
Platform to Platform project lightening talk
Hazel Hall
 
Undertaking a part-time LIS PhD: 10 tips in 20 minutes
Undertaking a part-time LIS PhD: 10 tips in 20 minutesUndertaking a part-time LIS PhD: 10 tips in 20 minutes
Undertaking a part-time LIS PhD: 10 tips in 20 minutes
Hazel Hall
 
Platform to Platform: initial findings from the empirical study
Platform to Platform: initial findings from the empirical studyPlatform to Platform: initial findings from the empirical study
Platform to Platform: initial findings from the empirical study
Hazel Hall
 
Digital options: an assessment of audience engagement with a digitised set of...
Digital options: an assessment of audience engagement with a digitised set of...Digital options: an assessment of audience engagement with a digitised set of...
Digital options: an assessment of audience engagement with a digitised set of...
Hazel Hall
 
Using a multi-location, longitudinal focus group method to conduct qualitativ...
Using a multi-location, longitudinal focus group method to conduct qualitativ...Using a multi-location, longitudinal focus group method to conduct qualitativ...
Using a multi-location, longitudinal focus group method to conduct qualitativ...
Hazel Hall
 
Introduction to RIVAL event 4
Introduction to RIVAL event 4Introduction to RIVAL event 4
Introduction to RIVAL event 4
Hazel Hall
 
Introduction to RIVAL event 3
Introduction to RIVAL event 3Introduction to RIVAL event 3
Introduction to RIVAL event 3
Hazel Hall
 
Research, impact, value and library and information science (RIVAL): developm...
Research, impact, value and library and information science (RIVAL): developm...Research, impact, value and library and information science (RIVAL): developm...
Research, impact, value and library and information science (RIVAL): developm...
Hazel Hall
 
Collaboration and networking: learning from DREaM and RIVAL
Collaboration and networking: learning from DREaM and RIVALCollaboration and networking: learning from DREaM and RIVAL
Collaboration and networking: learning from DREaM and RIVAL
Hazel Hall
 
Research into Practice case study 2: Library linked data implementations an...
	Research into Practice case study 2:  Library linked data implementations an...	Research into Practice case study 2:  Library linked data implementations an...
Research into Practice case study 2: Library linked data implementations an...
Hazel Hall
 
Catalysing research into practice from the ground up
Catalysing research into practice from the ground upCatalysing research into practice from the ground up
Catalysing research into practice from the ground up
Hazel Hall
 
Introduction to RIVAL event 2
Introduction to RIVAL event 2Introduction to RIVAL event 2
Introduction to RIVAL event 2
Hazel Hall
 
Ambitions for the RIVAL network
Ambitions for the RIVAL networkAmbitions for the RIVAL network
Ambitions for the RIVAL network
Hazel Hall
 
Scotland's school library strategy: advocacy and impact by Martina McChrystal
Scotland's school library strategy: advocacy and impact by Martina McChrystalScotland's school library strategy: advocacy and impact by Martina McChrystal
Scotland's school library strategy: advocacy and impact by Martina McChrystal
Hazel Hall
 
Introduction to RIVAL event 1
Introduction to RIVAL event 1Introduction to RIVAL event 1
Introduction to RIVAL event 1
Hazel Hall
 
Research Impact and Value in LIS: poster presented at Edge 2019
Research Impact and Value in LIS: poster presented at Edge 2019Research Impact and Value in LIS: poster presented at Edge 2019
Research Impact and Value in LIS: poster presented at Edge 2019
Hazel Hall
 
Participatory Budgeting, São Paulo, Brazil
Participatory Budgeting, São Paulo, BrazilParticipatory Budgeting, São Paulo, Brazil
Participatory Budgeting, São Paulo, Brazil
Hazel Hall
 
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slidesResearch Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
Hazel Hall
 

More from Hazel Hall (20)

Preparation of the PhD thesis for examination
Preparation of the PhD thesis for examinationPreparation of the PhD thesis for examination
Preparation of the PhD thesis for examination
 
RIVAL 2019-21: Did we get it right?
RIVAL 2019-21: Did we get it right? RIVAL 2019-21: Did we get it right?
RIVAL 2019-21: Did we get it right?
 
Platform to Platform project lightening talk
Platform to Platform project lightening talkPlatform to Platform project lightening talk
Platform to Platform project lightening talk
 
Undertaking a part-time LIS PhD: 10 tips in 20 minutes
Undertaking a part-time LIS PhD: 10 tips in 20 minutesUndertaking a part-time LIS PhD: 10 tips in 20 minutes
Undertaking a part-time LIS PhD: 10 tips in 20 minutes
 
Platform to Platform: initial findings from the empirical study
Platform to Platform: initial findings from the empirical studyPlatform to Platform: initial findings from the empirical study
Platform to Platform: initial findings from the empirical study
 
Digital options: an assessment of audience engagement with a digitised set of...
Digital options: an assessment of audience engagement with a digitised set of...Digital options: an assessment of audience engagement with a digitised set of...
Digital options: an assessment of audience engagement with a digitised set of...
 
Using a multi-location, longitudinal focus group method to conduct qualitativ...
Using a multi-location, longitudinal focus group method to conduct qualitativ...Using a multi-location, longitudinal focus group method to conduct qualitativ...
Using a multi-location, longitudinal focus group method to conduct qualitativ...
 
Introduction to RIVAL event 4
Introduction to RIVAL event 4Introduction to RIVAL event 4
Introduction to RIVAL event 4
 
Introduction to RIVAL event 3
Introduction to RIVAL event 3Introduction to RIVAL event 3
Introduction to RIVAL event 3
 
Research, impact, value and library and information science (RIVAL): developm...
Research, impact, value and library and information science (RIVAL): developm...Research, impact, value and library and information science (RIVAL): developm...
Research, impact, value and library and information science (RIVAL): developm...
 
Collaboration and networking: learning from DREaM and RIVAL
Collaboration and networking: learning from DREaM and RIVALCollaboration and networking: learning from DREaM and RIVAL
Collaboration and networking: learning from DREaM and RIVAL
 
Research into Practice case study 2: Library linked data implementations an...
	Research into Practice case study 2:  Library linked data implementations an...	Research into Practice case study 2:  Library linked data implementations an...
Research into Practice case study 2: Library linked data implementations an...
 
Catalysing research into practice from the ground up
Catalysing research into practice from the ground upCatalysing research into practice from the ground up
Catalysing research into practice from the ground up
 
Introduction to RIVAL event 2
Introduction to RIVAL event 2Introduction to RIVAL event 2
Introduction to RIVAL event 2
 
Ambitions for the RIVAL network
Ambitions for the RIVAL networkAmbitions for the RIVAL network
Ambitions for the RIVAL network
 
Scotland's school library strategy: advocacy and impact by Martina McChrystal
Scotland's school library strategy: advocacy and impact by Martina McChrystalScotland's school library strategy: advocacy and impact by Martina McChrystal
Scotland's school library strategy: advocacy and impact by Martina McChrystal
 
Introduction to RIVAL event 1
Introduction to RIVAL event 1Introduction to RIVAL event 1
Introduction to RIVAL event 1
 
Research Impact and Value in LIS: poster presented at Edge 2019
Research Impact and Value in LIS: poster presented at Edge 2019Research Impact and Value in LIS: poster presented at Edge 2019
Research Impact and Value in LIS: poster presented at Edge 2019
 
Participatory Budgeting, São Paulo, Brazil
Participatory Budgeting, São Paulo, BrazilParticipatory Budgeting, São Paulo, Brazil
Participatory Budgeting, São Paulo, Brazil
 
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slidesResearch Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
Research Impact Value and LIS, Edinburgh 11th July 2018: speaker slides
 

Project output versus influence in practice : impact as a dimension of research quality

  • 1. Project output versus influence in practice Impact as a dimension of research quality 6th International Conference on Evidence based Librarianship and Information Practice Keynote presentation by Professor Hazel Hall, Director, Centre for Social Informatics, 30th June 2011
  • 3.
  • 4. If you are tweeting this If you are tweeting this session, please refer to me session, please refer to me as @hazelh. Thank you! as @hazelh. Thank you!
  • 5. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 6. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 7. Impact themes in LIS literature Impact themes in LIS literature 1. Evaluation of services – general use (debated) 1. Evaluation of services – general use (debated)
  • 8. Impact themes in LIS literature Impact themes in LIS literature 2. Evaluation of service impact on end user behaviours 2. Evaluation of service impact on end user behaviours
  • 9. Impact themes in LIS Impact themes in LIS literature literature 3. Evaluation of services as 3. Evaluation of services as related to specific related to specific outcomes outcomes
  • 10. Impact themes in LIS literature Impact themes in LIS literature 4. How to evaluate services 4. How to evaluate services
  • 11. Impact themes in LIS literature Impact themes in LIS literature 5. Bibliometrics 5. Bibliometrics
  • 12. Impact themes in LIS literature Impact themes in LIS literature 5. Bibliometrics – publication impacts 5. Bibliometrics – publication impacts
  • 13. Impact themes in LIS literature Impact themes in LIS literature 6. Bibliometrics research evaluation 6. Bibliometrics research evaluation We can stand on the shoulders of giants We can stand on the shoulders of giants because librarians and information because librarians and information scientists have maintained a curated record scientists have maintained a curated record
  • 14. Impact themes in LIS literature Impact themes in LIS literature 7. LIS research output  LIS practice 7. LIS research output  LIS practice
  • 15. Impact themes in LIS literature Impact themes in LIS literature 7. LIS research output  LIS practice 7. LIS research output  LIS practice
  • 16. Impact themes in LIS literature Impact themes in LIS literature 8. LIS practice  LIS research 8. LIS practice  LIS research (Brookes, 1976) (Brookes, 1976)
  • 17. Impact routes vary Impact routes vary Cumulative and indirect effect of Cumulative and indirect effect of exposure to research output exposure to research output Direct policy mandate that Direct policy mandate that derives from a particular study derives from a particular study
  • 18. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 19. Measurement challenge 1 Measurement challenge 1 Attribution, and cause and effect, are Attribution, and cause and effect, are not always accurate nor obvious not always accurate nor obvious
  • 20. Greer Garson Greer Garson Best actress 1942 for role in Mrs Miniver Best actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds Acceptance speech 5 minutes 30 seconds Measurement challenge 1 Measurement challenge 1 Attribution, and cause and effect, are Attribution, and cause and effect, are not always accurate nor obvious not always accurate nor obvious
  • 21. Measurement challenge 2 Measurement challenge 2 Sources of mediated training material are Sources of mediated training material are not always appreciated by practitioners not always appreciated by practitioners
  • 22. Measurement challenge 3 Measurement challenge 3 End-users are not normally aware of research End-users are not normally aware of research effort to support services design and delivery effort to support services design and delivery
  • 23. Measurement challenge 4 Measurement challenge 4 Certain types of research output attract more Certain types of research output attract more attention than others attention than others
  • 24. Measurement challenge 5 Measurement challenge 5 The impact of some is overwhelming The impact of some is overwhelming (and disproportionate) (and disproportionate)
  • 25. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 26. Practitioner interest in research output Practitioner interest in research output Increases value of research undertaken Increases value of research undertaken Narrows research-practice gap  Narrows research-practice gap  benefits health of the profession benefits health of the profession
  • 27. Local contexts influence attitudes Local contexts influence attitudes towards impact measurement towards impact measurement REF2014 in the UK REF2014 in the UK
  • 28. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 29. NB presented here are strong NB presented here are strong messages that have emerged from messages that have emerged from RiLIES project empirical data RiLIES project empirical data analysed to date analysed to date http://lisresearch.org/rilies-project/ http://lisresearch.org/rilies-project/
  • 30. Literature review; poll; focus Literature review; poll; focus groups; case studies; survey groups; case studies; survey
  • 31. Research ideas for impact in practice Research ideas for impact in practice Focus on research areas of direct relevance Focus on research areas of direct relevance to practitioners, both in terms of subject to practitioners, both in terms of subject and context -- time, place, sector and context time, place, sector (Perth and London focus groups) (Perth and London focus groups)
  • 32. Research approach for impact in practice Research approach for impact in practice Deploy action research approach Deploy action research approach Mix in development and engagement, e.g. capacity Mix in development and engagement, e.g. capacity building as hook, workshops throughout project building as hook, workshops throughout project Encourage community creation around project for Encourage community creation around project for sustainability sustainability Involve high profile, prestigious partners Involve high profile, prestigious partners (Evalued, Evaluating the impact of clinical librarian (Evalued, Evaluating the impact of clinical librarian services, Open to all case studies) services, Open to all case studies)
  • 33. Project scale and diversity for impact in practice Project scale and diversity for impact in practice Data collection needs to be on a scale for project Data collection needs to be on a scale for project recommendations to be applicable elsewhere recommendations to be applicable elsewhere Ensure that projects are outward-looking to Ensure that projects are outward-looking to safeguard against accusations of bias safeguard against accusations of bias (Perth focus group) (Perth focus group)
  • 34. Face-to-face dissemination for impact in practice Face-to-face dissemination for impact in practice Exploit face-to-face dissemination as the practitioners’ Exploit face-to-face dissemination as the practitioners’ favoured route favoured route Target dissemination according to career stage preferences Target dissemination according to career stage preferences Invest project budget in training activities for project legacy Invest project budget in training activities for project legacy Launch events effective for policy maker target audiences Launch events effective for policy maker target audiences (Poll, focus groups, case studies) (Poll, focus groups, case studies)
  • 35. Accessible textual sources for impact in practice Accessible textual sources for impact in practice Tailor a variety of reports according to intended audiences: Tailor a variety of reports according to intended audiences: tools they can access, career stage, sector… tools they can access, career stage, sector… Make reports accessible: easy to locate e.g. open access; Make reports accessible: easy to locate e.g. open access; understandable, e.g. possible to act on project outcomes understandable, e.g. possible to act on project outcomes Work with profile dissemination partners (e.g. HEFCE, JISC, Work with profile dissemination partners (e.g. HEFCE, JISC, OCLC, Pew, RIN, SCONUL) to enhance authority of work and OCLC, Pew, RIN, SCONUL) to enhance authority of work and increase its reach increase its reach Ensure work is picked up by digest/alert services and bloggers Ensure work is picked up by digest/alert services and bloggers
  • 36. http://www.twitter.com/LISResearch http://www.twitter.com/LISResearch provides an alert services provides an alert services Conference registration open Conference registration open Training day announcement Training day announcement Publication of research report Publication of research report Study day announcement Study day announcement Conference registration to close Conference registration to close Conference registration open Conference registration open Study day announcement Study day announcement Publication of blog post Publication of blog post
  • 37. Exploit network hubs Exploit network hubs 5 key individuals in UK 5 key individuals in UK emerge from RiLIES project… emerge from RiLIES project…
  • 38. 5 UK research connectors 5 UK research connectors 1. Rhona Arthur @RhonaEA 1. Rhona Arthur @RhonaEA
  • 39. 5 UK research connectors 5 UK research connectors 2. Andrew Booth @scharrlock 2. Andrew Booth @scharrlock
  • 40. 5 UK research connectors 5 UK research connectors 3. Phil Bradley @Philbradley 3. Phil Bradley @Philbradley
  • 41. 5 UK research connectors 5 UK research connectors 4. Alison Brettle @BrettleAli 4. Alison Brettle @BrettleAli
  • 42. 5 UK research 5 UK research connectors connectors 5. Hazel Hall @hazelh 5. Hazel Hall @hazelh
  • 43. Face-to-face dissemination for impact in practice Face-to-face dissemination for impact in practice What happens when formal opportunities for face-to- What happens when formal opportunities for face-to- face dissemination are restricted: face dissemination are restricted: training/travel budgets cut? training/travel budgets cut? reduced professional memberships? reduced professional memberships? … and when research infrastructures are fractured? This … and when research infrastructures are fractured? This has been addressed in UK in the formation of the has been addressed in UK in the formation of the Library and Information Science Research Coalition Library and Information Science Research Coalition
  • 44. Researcher priority Researcher priority Present research output in an accessible Present research output in an accessible way way Service manager priority Service manager priority Facilitate research output consumption, Facilitate research output consumption, e.g. reverse mentoring, job descriptions, e.g. reverse mentoring, job descriptions, reporting back from conferences reporting back from conferences Course/accreditation priority Course/accreditation priority Application of research in practice Application of research in practice
  • 45. A greater role for social media? A greater role for social media? Currently divides the practitioner Currently divides the practitioner community, e.g. blogs very popular with community, e.g. blogs very popular with academic librarians but inaccessible to academic librarians but inaccessible to some others; Twitter essential to some, some others; Twitter essential to some, irrelevant to others irrelevant to others http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/
  • 46. A greater role for self-organising A greater role for self-organising networks as communities of practice? networks as communities of practice? (with technical infrastructure provided by (with technical infrastructure provided by social media) social media)
  • 47. Research funding to support researcher- Research funding to support researcher- practitioner links practitioner links http://www.lisresearch.org/dream-project http://www.lisresearch.org/dream-project
  • 48. DREaM project launch conference DREaM project launch conference Tuesday 19th July, London Tuesday 19th July, London Places still available Places still available
  • 49. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach  Preliminary findings of RiLIES project dominated by influence of dissemination strategy
  • 50. Bringing it back to Salford: anyone recognise these lads?
  • 51. Bringing it back to Salford: anyone recognise these lads?
  • 52. What difference does it make? (1984)
  • 53.
  • 54. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  • 55. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  • 56. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  • 57. Photo credits Image: 'I Can Gather All the News I+Need+on+the+Weather+Report' Image: 'Greer Garson & Robert Young' http://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154 Image: 'These Books are Sweet' Image: ‘Waiting for the library to open’ http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set- 72157602122007409 Image: ‘Radar Graph – Library Usage v Final Grades' http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: 'BC007' http://www.flickr.com/photos/51548378@N05/4745027706 Image: '50lb weights' http://www.flickr.com/photos/62327186@N00/2185667691 Image: 'Money to burn' http://www.flickr.com/photos/8363329@N05/811315055 Image: 'Article-level metrics at PLoS' http://www.flickr.com/photos/14829735@N00/3929622080 Image: 'Warhol's Light Bulbs' http://www.flickr.com/photos/66814335@N00/3036254720 Image: '365: day 119' http://www.flickr.com/photos/33563858@N00/2451800342 Image: 'Cheers' http://www.flickr.com/photos/71477195@N00/204879104 Image: 'The locker misery' http://www.flickr.com/photos/33685308@N00/513100970 Image: 'P1010121' http://www.flickr.com/photos/67614763@N00/4304374413 Image: '90" engine valves' http://www.flickr.com/photos/61132483@N00/2975215 Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog' http://www.flickr.com/photos/49503019876@N01/1824234195 Image: 'Oscar' http://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead) by Stephen Wright Image: 'ed vaizey' http://www.flickr.com/photos/25414047@N00/4065621328
  • 58. Reference & acknowledgements  Brookes, B.C. (1976). A new paradigm for information science. The Information Scientist, 10(3), 103-12.  Peter Cruickshank, Ella Taylor-Smith, Jenny Gebel (RiLIES project team)  Stephanie Kenna (LIS Research Coalition)  All those who have contributed data to date