Your SlideShare is downloading. ×
Project output versus influence in practice: impact as a dimension of research quality
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Project output versus influence in practice : impact as a dimension of research quality

183
views

Published on

Hazel Hall's invited keynote paper presented at the 6th International Evidence Based Library and Information Practice (EBLIP6) conference, Manchester, 30 June 2011. A journal article that draws on the …

Hazel Hall's invited keynote paper presented at the 6th International Evidence Based Library and Information Practice (EBLIP6) conference, Manchester, 30 June 2011. A journal article that draws on the some of the content of this presentation is also available at http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/12138/9373


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
183
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Project output versus influence inpracticeImpact as a dimension of researchquality6th International Conference on Evidence based Librarianship and InformationPracticeKeynote presentation by Professor Hazel Hall, Director, Centre for SocialInformatics, 30th June 2011
  • 2. http://lisresearch.orghttp://lisresearch.org
  • 3. If you are tweeting thisIf you are tweeting thissession, please refer to mesession, please refer to meas @hazelh. Thank you!as @hazelh. Thank you!
  • 4. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 5. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 6. Impact themes in LIS literatureImpact themes in LIS literature1. Evaluation of services – general use (debated)1. Evaluation of services – general use (debated)
  • 7. Impact themes in LIS literatureImpact themes in LIS literature2. Evaluation of service impact on end user behaviours2. Evaluation of service impact on end user behaviours
  • 8. Impact themes in LISImpact themes in LISliteratureliterature3. Evaluation of services as3. Evaluation of services asrelated to specificrelated to specificoutcomesoutcomes
  • 9. Impact themes in LIS literatureImpact themes in LIS literature4. How to evaluate services4. How to evaluate services
  • 10. Impact themes in LIS literatureImpact themes in LIS literature5. Bibliometrics5. Bibliometrics
  • 11. Impact themes in LIS literatureImpact themes in LIS literature5. Bibliometrics – publication impacts5. Bibliometrics – publication impacts
  • 12. Impact themes in LIS literatureImpact themes in LIS literature6. Bibliometrics research evaluation6. Bibliometrics research evaluationWe can stand on the shoulders of giantsWe can stand on the shoulders of giantsbecause librarians and informationbecause librarians and informationscientists have maintained a curated recordscientists have maintained a curated record
  • 13. Impact themes in LIS literatureImpact themes in LIS literature7. LIS research output  LIS practice7. LIS research output  LIS practice
  • 14. Impact themes in LIS literatureImpact themes in LIS literature7. LIS research output  LIS practice7. LIS research output  LIS practice
  • 15. Impact themes in LIS literatureImpact themes in LIS literature8. LIS practice  LIS research8. LIS practice  LIS research(Brookes, 1976)(Brookes, 1976)
  • 16. Impact routes varyImpact routes varyCumulative and indirect effect ofCumulative and indirect effect ofexposure to research outputexposure to research outputDirect policy mandate thatDirect policy mandate thatderives from a particular studyderives from a particular study
  • 17. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 18. Measurement challenge 1Measurement challenge 1Attribution, and cause and effect, areAttribution, and cause and effect, arenot always accurate nor obviousnot always accurate nor obvious
  • 19. Greer Garson Greer Garson Best actress 1942 for role in Mrs Miniver Best actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds Acceptance speech 5 minutes 30 secondsMeasurement challenge 1Measurement challenge 1Attribution, and cause and effect, areAttribution, and cause and effect, arenot always accurate nor obviousnot always accurate nor obvious
  • 20. Measurement challenge 2Measurement challenge 2Sources of mediated training material areSources of mediated training material arenot always appreciated by practitionersnot always appreciated by practitioners
  • 21. Measurement challenge 3Measurement challenge 3End-users are not normally aware of researchEnd-users are not normally aware of researcheffort to support services design and deliveryeffort to support services design and delivery
  • 22. Measurement challenge 4Measurement challenge 4Certain types of research output attract moreCertain types of research output attract moreattention than othersattention than others
  • 23. Measurement challenge 5Measurement challenge 5The impact of some is overwhelmingThe impact of some is overwhelming(and disproportionate)(and disproportionate)
  • 24. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 25. Practitioner interest in research outputPractitioner interest in research outputIncreases value of research undertakenIncreases value of research undertakenNarrows research-practice gap Narrows research-practice gap benefits health of the professionbenefits health of the profession
  • 26. Local contexts influence attitudesLocal contexts influence attitudestowards impact measurementtowards impact measurementREF2014 in the UKREF2014 in the UK
  • 27. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  • 28. NB presented here are strongNB presented here are strongmessages that have emerged frommessages that have emerged fromRiLIES project empirical dataRiLIES project empirical dataanalysed to dateanalysed to datehttp://lisresearch.org/rilies-project/http://lisresearch.org/rilies-project/
  • 29. Literature review; poll; focusLiterature review; poll; focusgroups; case studies; surveygroups; case studies; survey
  • 30. Research ideas for impact in practiceResearch ideas for impact in practiceFocus on research areas of direct relevanceFocus on research areas of direct relevanceto practitioners, both in terms of subjectto practitioners, both in terms of subjectand context -- time, place, sectorand context time, place, sector(Perth and London focus groups)(Perth and London focus groups)
  • 31. Research approach for impact in practiceResearch approach for impact in practiceDeploy action research approachDeploy action research approachMix in development and engagement, e.g. capacityMix in development and engagement, e.g. capacitybuilding as hook, workshops throughout projectbuilding as hook, workshops throughout projectEncourage community creation around project forEncourage community creation around project forsustainabilitysustainabilityInvolve high profile, prestigious partnersInvolve high profile, prestigious partners(Evalued, Evaluating the impact of clinical librarian(Evalued, Evaluating the impact of clinical librarianservices, Open to all case studies)services, Open to all case studies)
  • 32. Project scale and diversity for impact in practiceProject scale and diversity for impact in practiceData collection needs to be on a scale for projectData collection needs to be on a scale for projectrecommendations to be applicable elsewhererecommendations to be applicable elsewhereEnsure that projects are outward-looking toEnsure that projects are outward-looking tosafeguard against accusations of biassafeguard against accusations of bias(Perth focus group)(Perth focus group)
  • 33. Face-to-face dissemination for impact in practiceFace-to-face dissemination for impact in practiceExploit face-to-face dissemination as the practitioners’Exploit face-to-face dissemination as the practitioners’favoured routefavoured routeTarget dissemination according to career stage preferencesTarget dissemination according to career stage preferencesInvest project budget in training activities for project legacyInvest project budget in training activities for project legacyLaunch events effective for policy maker target audiencesLaunch events effective for policy maker target audiences(Poll, focus groups, case studies)(Poll, focus groups, case studies)
  • 34. Accessible textual sources for impact in practiceAccessible textual sources for impact in practiceTailor a variety of reports according to intended audiences:Tailor a variety of reports according to intended audiences:tools they can access, career stage, sector…tools they can access, career stage, sector…Make reports accessible: easy to locate e.g. open access;Make reports accessible: easy to locate e.g. open access;understandable, e.g. possible to act on project outcomesunderstandable, e.g. possible to act on project outcomesWork with profile dissemination partners (e.g. HEFCE, JISC, Work with profile dissemination partners (e.g. HEFCE, JISC,OCLC, Pew, RIN, SCONUL) to enhance authority of work and OCLC, Pew, RIN, SCONUL) to enhance authority of work andincrease its reach increase its reachEnsure work is picked up by digest/alert services and bloggersEnsure work is picked up by digest/alert services and bloggers
  • 35. http://www.twitter.com/LISResearchhttp://www.twitter.com/LISResearchprovides an alert servicesprovides an alert services Conference registration open Conference registration open Training day announcement Training day announcement Publication of research report Publication of research report Study day announcement Study day announcement Conference registration to close Conference registration to close Conference registration open Conference registration open Study day announcement Study day announcement Publication of blog post Publication of blog post
  • 36. Exploit network hubsExploit network hubs5 key individuals in UK5 key individuals in UKemerge from RiLIES project…emerge from RiLIES project…
  • 37. 5 UK research connectors5 UK research connectors1. Rhona Arthur @RhonaEA1. Rhona Arthur @RhonaEA
  • 38. 5 UK research connectors5 UK research connectors2. Andrew Booth @scharrlock2. Andrew Booth @scharrlock
  • 39. 5 UK research connectors5 UK research connectors3. Phil Bradley @Philbradley3. Phil Bradley @Philbradley
  • 40. 5 UK research connectors5 UK research connectors4. Alison Brettle @BrettleAli4. Alison Brettle @BrettleAli
  • 41. 5 UK research5 UK researchconnectorsconnectors5. Hazel Hall @hazelh5. Hazel Hall @hazelh
  • 42. Face-to-face dissemination for impact in practiceFace-to-face dissemination for impact in practiceWhat happens when formal opportunities for face-to-What happens when formal opportunities for face-to-face dissemination are restricted:face dissemination are restricted: training/travel budgets cut? training/travel budgets cut? reduced professional memberships? reduced professional memberships?… and when research infrastructures are fractured? This… and when research infrastructures are fractured? Thishas been addressed in UK in the formation of thehas been addressed in UK in the formation of theLibrary and Information Science Research CoalitionLibrary and Information Science Research Coalition
  • 43. Researcher priorityResearcher priorityPresent research output in an accessiblePresent research output in an accessiblewaywayService manager priorityService manager priorityFacilitate research output consumption,Facilitate research output consumption,e.g. reverse mentoring, job descriptions,e.g. reverse mentoring, job descriptions,reporting back from conferencesreporting back from conferencesCourse/accreditation priorityCourse/accreditation priorityApplication of research in practiceApplication of research in practice
  • 44. A greater role for social media? A greater role for social media? Currently divides the practitioner Currently divides the practitioner community, e.g. blogs very popular with community, e.g. blogs very popular with academic librarians but inaccessible to academic librarians but inaccessible to some others; Twitter essential to some, some others; Twitter essential to some, irrelevant to others irrelevant to othershttp://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/
  • 45. A greater role for self-organisingA greater role for self-organisingnetworks as communities of practice?networks as communities of practice?(with technical infrastructure provided by(with technical infrastructure provided bysocial media)social media)
  • 46. Research funding to support researcher-Research funding to support researcher-practitioner linkspractitioner linkshttp://www.lisresearch.org/dream-projecthttp://www.lisresearch.org/dream-project
  • 47. DREaM project launch conferenceDREaM project launch conferenceTuesday 19th July, LondonTuesday 19th July, LondonPlaces still availablePlaces still available
  • 48. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach  Preliminary findings of RiLIES project dominated by influence of dissemination strategy
  • 49. Bringing it back to Salford:anyone recognise these lads?
  • 50. Bringing it back to Salford:anyone recognise these lads?
  • 51. What difference does it make? (1984)
  • 52. Playlist 20 March 1984Playlist 20 March 1984Hand in glove; Still ill; Heaven knows I’mHand in glove; Still ill; Heaven knows I’mmiserable now; This charming man; Girl afraid;miserable now; This charming man; Girl afraid;Pretty girls make graves; This night has openedPretty girls make graves; This night has openedmy eyes; Barbarism begins at home; Whatmy eyes; Barbarism begins at home; Whatdifference does it make? II don’t owe youdifference does it make? don’t owe youanything; Handsome devil; You’ve gotanything; Handsome devil; You’ve goteverything now; These things take timeeverything now; These things take time
  • 53. Playlist 20 March 1984Playlist 20 March 1984Hand in glove; Still ill; Heaven knows I’mHand in glove; Still ill; Heaven knows I’mmiserable now; This charming man; Girl afraid;miserable now; This charming man; Girl afraid;Pretty girls make graves; This night has openedPretty girls make graves; This night has openedmy eyes; Barbarism begins at home; Whatmy eyes; Barbarism begins at home; Whatdifference does it make? II don’t owe youdifference does it make? don’t owe youanything; Handsome devil; You’ve gotanything; Handsome devil; You’ve goteverything now; These things take timeeverything now; These things take time
  • 54. Playlist 20 March 1984Playlist 20 March 1984Hand in glove; Still ill; Heaven knows I’mHand in glove; Still ill; Heaven knows I’mmiserable now; This charming man; Girl afraid;miserable now; This charming man; Girl afraid;Pretty girls make graves; This night has openedPretty girls make graves; This night has openedmy eyes; Barbarism begins at home; Whatmy eyes; Barbarism begins at home; Whatdifference does it make? II don’t owe youdifference does it make? don’t owe youanything; Handsome devil; You’ve gotanything; Handsome devil; You’ve goteverything now; These things take timeeverything now; These things take time
  • 55. Photo creditsImage: I Can Gather All the News I+Need+on+the+Weather+Report Image: Greer Garson & Robert Younghttp://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154Image: These Books are Sweet Image: ‘Waiting for the library to open’http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set- 72157602122007409Image: ‘Radar Graph – Library Usage v Final Gradeshttp://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: BC007 http://www.flickr.com/photos/51548378@N05/4745027706Image: 50lb weightshttp://www.flickr.com/photos/62327186@N00/2185667691 Image: Money to burn http://www.flickr.com/photos/8363329@N05/811315055Image: Article-level metrics at PLoShttp://www.flickr.com/photos/14829735@N00/3929622080 Image: Warhols Light Bulbs http://www.flickr.com/photos/66814335@N00/3036254720Image: 365: day 119http://www.flickr.com/photos/33563858@N00/2451800342 Image: Cheers http://www.flickr.com/photos/71477195@N00/204879104Image: The locker miseryhttp://www.flickr.com/photos/33685308@N00/513100970 Image: P1010121 http://www.flickr.com/photos/67614763@N00/4304374413Image: 90" engine valveshttp://www.flickr.com/photos/61132483@N00/2975215 Image: My social Network on Flickr, Facebook, Twitter+and+MyblogLog http://www.flickr.com/photos/49503019876@N01/1824234195Image: Oscarhttp://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead) by Stephen WrightImage: ed vaizeyhttp://www.flickr.com/photos/25414047@N00/4065621328
  • 56. Reference & acknowledgements Brookes, B.C. (1976). A new paradigm for information science. The Information Scientist, 10(3), 103-12. Peter Cruickshank, Ella Taylor-Smith, Jenny Gebel (RiLIES project team) Stephanie Kenna (LIS Research Coalition) All those who have contributed data to date
  • 57. Professor Hazel Hallh.hall@napier.ac.uk @hazelh