Successfully reported this slideshow.
Your SlideShare is downloading. ×

Project output versus influence in practice : impact as a dimension of research quality

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 59 Ad

Project output versus influence in practice : impact as a dimension of research quality

Download to read offline

Hazel Hall's invited keynote paper presented at the 6th International Evidence Based Library and Information Practice (EBLIP6) conference, Manchester, 30 June 2011. A journal article that draws on the some of the content of this presentation is also available at http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/12138/9373

Hazel Hall's invited keynote paper presented at the 6th International Evidence Based Library and Information Practice (EBLIP6) conference, Manchester, 30 June 2011. A journal article that draws on the some of the content of this presentation is also available at http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/12138/9373

Advertisement
Advertisement

More Related Content

Similar to Project output versus influence in practice : impact as a dimension of research quality (20)

Advertisement

More from Hazel Hall (20)

Project output versus influence in practice : impact as a dimension of research quality

  1. 1. Project output versus influence in practice Impact as a dimension of research quality 6th International Conference on Evidence based Librarianship and Information Practice Keynote presentation by Professor Hazel Hall, Director, Centre for Social Informatics, 30th June 2011
  2. 2. http://lisresearch.org http://lisresearch.org
  3. 3. If you are tweeting this If you are tweeting this session, please refer to me session, please refer to me as @hazelh. Thank you! as @hazelh. Thank you!
  4. 4. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  5. 5. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  6. 6. Impact themes in LIS literature Impact themes in LIS literature 1. Evaluation of services – general use (debated) 1. Evaluation of services – general use (debated)
  7. 7. Impact themes in LIS literature Impact themes in LIS literature 2. Evaluation of service impact on end user behaviours 2. Evaluation of service impact on end user behaviours
  8. 8. Impact themes in LIS Impact themes in LIS literature literature 3. Evaluation of services as 3. Evaluation of services as related to specific related to specific outcomes outcomes
  9. 9. Impact themes in LIS literature Impact themes in LIS literature 4. How to evaluate services 4. How to evaluate services
  10. 10. Impact themes in LIS literature Impact themes in LIS literature 5. Bibliometrics 5. Bibliometrics
  11. 11. Impact themes in LIS literature Impact themes in LIS literature 5. Bibliometrics – publication impacts 5. Bibliometrics – publication impacts
  12. 12. Impact themes in LIS literature Impact themes in LIS literature 6. Bibliometrics research evaluation 6. Bibliometrics research evaluation We can stand on the shoulders of giants We can stand on the shoulders of giants because librarians and information because librarians and information scientists have maintained a curated record scientists have maintained a curated record
  13. 13. Impact themes in LIS literature Impact themes in LIS literature 7. LIS research output  LIS practice 7. LIS research output  LIS practice
  14. 14. Impact themes in LIS literature Impact themes in LIS literature 7. LIS research output  LIS practice 7. LIS research output  LIS practice
  15. 15. Impact themes in LIS literature Impact themes in LIS literature 8. LIS practice  LIS research 8. LIS practice  LIS research (Brookes, 1976) (Brookes, 1976)
  16. 16. Impact routes vary Impact routes vary Cumulative and indirect effect of Cumulative and indirect effect of exposure to research output exposure to research output Direct policy mandate that Direct policy mandate that derives from a particular study derives from a particular study
  17. 17. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  18. 18. Measurement challenge 1 Measurement challenge 1 Attribution, and cause and effect, are Attribution, and cause and effect, are not always accurate nor obvious not always accurate nor obvious
  19. 19. Greer Garson Greer Garson Best actress 1942 for role in Mrs Miniver Best actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds Acceptance speech 5 minutes 30 seconds Measurement challenge 1 Measurement challenge 1 Attribution, and cause and effect, are Attribution, and cause and effect, are not always accurate nor obvious not always accurate nor obvious
  20. 20. Measurement challenge 2 Measurement challenge 2 Sources of mediated training material are Sources of mediated training material are not always appreciated by practitioners not always appreciated by practitioners
  21. 21. Measurement challenge 3 Measurement challenge 3 End-users are not normally aware of research End-users are not normally aware of research effort to support services design and delivery effort to support services design and delivery
  22. 22. Measurement challenge 4 Measurement challenge 4 Certain types of research output attract more Certain types of research output attract more attention than others attention than others
  23. 23. Measurement challenge 5 Measurement challenge 5 The impact of some is overwhelming The impact of some is overwhelming (and disproportionate) (and disproportionate)
  24. 24. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  25. 25. Practitioner interest in research output Practitioner interest in research output Increases value of research undertaken Increases value of research undertaken Narrows research-practice gap  Narrows research-practice gap  benefits health of the profession benefits health of the profession
  26. 26. Local contexts influence attitudes Local contexts influence attitudes towards impact measurement towards impact measurement REF2014 in the UK REF2014 in the UK
  27. 27. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach
  28. 28. NB presented here are strong NB presented here are strong messages that have emerged from messages that have emerged from RiLIES project empirical data RiLIES project empirical data analysed to date analysed to date http://lisresearch.org/rilies-project/ http://lisresearch.org/rilies-project/
  29. 29. Literature review; poll; focus Literature review; poll; focus groups; case studies; survey groups; case studies; survey
  30. 30. Research ideas for impact in practice Research ideas for impact in practice Focus on research areas of direct relevance Focus on research areas of direct relevance to practitioners, both in terms of subject to practitioners, both in terms of subject and context -- time, place, sector and context time, place, sector (Perth and London focus groups) (Perth and London focus groups)
  31. 31. Research approach for impact in practice Research approach for impact in practice Deploy action research approach Deploy action research approach Mix in development and engagement, e.g. capacity Mix in development and engagement, e.g. capacity building as hook, workshops throughout project building as hook, workshops throughout project Encourage community creation around project for Encourage community creation around project for sustainability sustainability Involve high profile, prestigious partners Involve high profile, prestigious partners (Evalued, Evaluating the impact of clinical librarian (Evalued, Evaluating the impact of clinical librarian services, Open to all case studies) services, Open to all case studies)
  32. 32. Project scale and diversity for impact in practice Project scale and diversity for impact in practice Data collection needs to be on a scale for project Data collection needs to be on a scale for project recommendations to be applicable elsewhere recommendations to be applicable elsewhere Ensure that projects are outward-looking to Ensure that projects are outward-looking to safeguard against accusations of bias safeguard against accusations of bias (Perth focus group) (Perth focus group)
  33. 33. Face-to-face dissemination for impact in practice Face-to-face dissemination for impact in practice Exploit face-to-face dissemination as the practitioners’ Exploit face-to-face dissemination as the practitioners’ favoured route favoured route Target dissemination according to career stage preferences Target dissemination according to career stage preferences Invest project budget in training activities for project legacy Invest project budget in training activities for project legacy Launch events effective for policy maker target audiences Launch events effective for policy maker target audiences (Poll, focus groups, case studies) (Poll, focus groups, case studies)
  34. 34. Accessible textual sources for impact in practice Accessible textual sources for impact in practice Tailor a variety of reports according to intended audiences: Tailor a variety of reports according to intended audiences: tools they can access, career stage, sector… tools they can access, career stage, sector… Make reports accessible: easy to locate e.g. open access; Make reports accessible: easy to locate e.g. open access; understandable, e.g. possible to act on project outcomes understandable, e.g. possible to act on project outcomes Work with profile dissemination partners (e.g. HEFCE, JISC, Work with profile dissemination partners (e.g. HEFCE, JISC, OCLC, Pew, RIN, SCONUL) to enhance authority of work and OCLC, Pew, RIN, SCONUL) to enhance authority of work and increase its reach increase its reach Ensure work is picked up by digest/alert services and bloggers Ensure work is picked up by digest/alert services and bloggers
  35. 35. http://www.twitter.com/LISResearch http://www.twitter.com/LISResearch provides an alert services provides an alert services Conference registration open Conference registration open Training day announcement Training day announcement Publication of research report Publication of research report Study day announcement Study day announcement Conference registration to close Conference registration to close Conference registration open Conference registration open Study day announcement Study day announcement Publication of blog post Publication of blog post
  36. 36. Exploit network hubs Exploit network hubs 5 key individuals in UK 5 key individuals in UK emerge from RiLIES project… emerge from RiLIES project…
  37. 37. 5 UK research connectors 5 UK research connectors 1. Rhona Arthur @RhonaEA 1. Rhona Arthur @RhonaEA
  38. 38. 5 UK research connectors 5 UK research connectors 2. Andrew Booth @scharrlock 2. Andrew Booth @scharrlock
  39. 39. 5 UK research connectors 5 UK research connectors 3. Phil Bradley @Philbradley 3. Phil Bradley @Philbradley
  40. 40. 5 UK research connectors 5 UK research connectors 4. Alison Brettle @BrettleAli 4. Alison Brettle @BrettleAli
  41. 41. 5 UK research 5 UK research connectors connectors 5. Hazel Hall @hazelh 5. Hazel Hall @hazelh
  42. 42. Face-to-face dissemination for impact in practice Face-to-face dissemination for impact in practice What happens when formal opportunities for face-to- What happens when formal opportunities for face-to- face dissemination are restricted: face dissemination are restricted: training/travel budgets cut? training/travel budgets cut? reduced professional memberships? reduced professional memberships? … and when research infrastructures are fractured? This … and when research infrastructures are fractured? This has been addressed in UK in the formation of the has been addressed in UK in the formation of the Library and Information Science Research Coalition Library and Information Science Research Coalition
  43. 43. Researcher priority Researcher priority Present research output in an accessible Present research output in an accessible way way Service manager priority Service manager priority Facilitate research output consumption, Facilitate research output consumption, e.g. reverse mentoring, job descriptions, e.g. reverse mentoring, job descriptions, reporting back from conferences reporting back from conferences Course/accreditation priority Course/accreditation priority Application of research in practice Application of research in practice
  44. 44. A greater role for social media? A greater role for social media? Currently divides the practitioner Currently divides the practitioner community, e.g. blogs very popular with community, e.g. blogs very popular with academic librarians but inaccessible to academic librarians but inaccessible to some others; Twitter essential to some, some others; Twitter essential to some, irrelevant to others irrelevant to others http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/
  45. 45. A greater role for self-organising A greater role for self-organising networks as communities of practice? networks as communities of practice? (with technical infrastructure provided by (with technical infrastructure provided by social media) social media)
  46. 46. Research funding to support researcher- Research funding to support researcher- practitioner links practitioner links http://www.lisresearch.org/dream-project http://www.lisresearch.org/dream-project
  47. 47. DREaM project launch conference DREaM project launch conference Tuesday 19th July, London Tuesday 19th July, London Places still available Places still available
  48. 48. Key points  LIS impact takes a number of forms  Research impact measurement is tricky  Research impact measurement matters  Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach  Preliminary findings of RiLIES project dominated by influence of dissemination strategy
  49. 49. Bringing it back to Salford: anyone recognise these lads?
  50. 50. Bringing it back to Salford: anyone recognise these lads?
  51. 51. What difference does it make? (1984)
  52. 52. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  53. 53. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  54. 54. Playlist 20 March 1984 Playlist 20 March 1984 Hand in glove; Still ill; Heaven knows I’m Hand in glove; Still ill; Heaven knows I’m miserable now; This charming man; Girl afraid; miserable now; This charming man; Girl afraid; Pretty girls make graves; This night has opened Pretty girls make graves; This night has opened my eyes; Barbarism begins at home; What my eyes; Barbarism begins at home; What difference does it make? II don’t owe you difference does it make? don’t owe you anything; Handsome devil; You’ve got anything; Handsome devil; You’ve got everything now; These things take time everything now; These things take time
  55. 55. Photo credits Image: 'I Can Gather All the News I+Need+on+the+Weather+Report' Image: 'Greer Garson & Robert Young' http://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154 Image: 'These Books are Sweet' Image: ‘Waiting for the library to open’ http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set- 72157602122007409 Image: ‘Radar Graph – Library Usage v Final Grades' http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: 'BC007' http://www.flickr.com/photos/51548378@N05/4745027706 Image: '50lb weights' http://www.flickr.com/photos/62327186@N00/2185667691 Image: 'Money to burn' http://www.flickr.com/photos/8363329@N05/811315055 Image: 'Article-level metrics at PLoS' http://www.flickr.com/photos/14829735@N00/3929622080 Image: 'Warhol's Light Bulbs' http://www.flickr.com/photos/66814335@N00/3036254720 Image: '365: day 119' http://www.flickr.com/photos/33563858@N00/2451800342 Image: 'Cheers' http://www.flickr.com/photos/71477195@N00/204879104 Image: 'The locker misery' http://www.flickr.com/photos/33685308@N00/513100970 Image: 'P1010121' http://www.flickr.com/photos/67614763@N00/4304374413 Image: '90" engine valves' http://www.flickr.com/photos/61132483@N00/2975215 Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog' http://www.flickr.com/photos/49503019876@N01/1824234195 Image: 'Oscar' http://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead) by Stephen Wright Image: 'ed vaizey' http://www.flickr.com/photos/25414047@N00/4065621328
  56. 56. Reference & acknowledgements  Brookes, B.C. (1976). A new paradigm for information science. The Information Scientist, 10(3), 103-12.  Peter Cruickshank, Ella Taylor-Smith, Jenny Gebel (RiLIES project team)  Stephanie Kenna (LIS Research Coalition)  All those who have contributed data to date
  57. 57. Professor Hazel Hall h.hall@napier.ac.uk @hazelh

×