Published on

Claire Creaser (Director, LISU): Evaluation: with particular reference to public libraries

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Evaluation with particular reference to public libraries Claire Creaser
  2. 2. Introduction <ul><li>About LISU </li></ul><ul><li>Evaluation </li></ul><ul><ul><li>Evidence based librarianship </li></ul></ul><ul><ul><li>Project evaluation </li></ul></ul><ul><li>Examples from LISU’s portfolio </li></ul>9-7-10 LIRG AGM
  3. 3. About LISU <ul><li>Research unit </li></ul><ul><li>Team of researchers & statisticians </li></ul><ul><ul><li>With librarianship training & experience </li></ul></ul><ul><li>Involved in wide variety of: </li></ul><ul><ul><li>Consultancy </li></ul></ul><ul><ul><li>Training </li></ul></ul><ul><ul><li>Research </li></ul></ul>9-7-10 LIRG AGM
  4. 4. LISU support for management of LIS <ul><li>Benchmarking and statistical analysis </li></ul><ul><li>Range of library evaluations </li></ul><ul><ul><li>Options for assessing Digital Library performance at the National Library of Scotland </li></ul></ul><ul><ul><li>Case studies of successful primary school libraries </li></ul></ul><ul><ul><li>Benchmarking Senate House Library, University of London </li></ul></ul><ul><ul><li>Evaluation of Derwent Healthy Living Centre Library </li></ul></ul><ul><ul><li>... </li></ul></ul>9-7-10 LIRG AGM
  5. 5. Evaluation <ul><li>The process of making a judgment about the worth of something </li></ul><ul><ul><li>After considering it carefully </li></ul></ul><ul><li>Why evaluate? </li></ul><ul><ul><li>Better management </li></ul></ul><ul><ul><ul><li>Target setting </li></ul></ul></ul><ul><ul><ul><li>Managing change </li></ul></ul></ul><ul><ul><ul><li>Efficiency gains </li></ul></ul></ul><ul><li>Advocacy </li></ul><ul><ul><li>Developing strategy </li></ul></ul><ul><ul><li>Empowering staff </li></ul></ul>9-7-10 LIRG AGM
  6. 6. Evaluation Criteria <ul><li>Aims and objectives </li></ul><ul><li>Success indicators - </li></ul><ul><ul><li>New / improved </li></ul></ul><ul><ul><li>Added value </li></ul></ul><ul><li>Before and after </li></ul>9-7-10 LIRG AGM
  7. 7. Evidence based librarianship <ul><li>‘ An approach to information practice that promotes the collection, interpretation and integration of valid, important and applicable, user-reported, librarian-observed and research-derived evidence. The best available evidence …is applied to improve the quality of professional judgements’ (Brice et al ., 2005) </li></ul>9-7-10 LIRG AGM
  8. 8. Six practical steps of EBM <ul><li>Defining the problem </li></ul><ul><li>Finding evidence </li></ul><ul><ul><li>What evidence do you really need to do this? </li></ul></ul><ul><ul><li>How will you collect / source it? </li></ul></ul><ul><ul><li>Very little evidence is useful on its own! </li></ul></ul><ul><li>Appraising the evidence </li></ul><ul><ul><li>Is the evidence reliable and relevant? </li></ul></ul><ul><ul><li>How will you interpret the information? </li></ul></ul>9-7-10 LIRG AGM
  9. 9. Six practical steps of EBM 9-7-10 LIRG AGM
  10. 10. Project Evaluation <ul><li>Fundamental Questions:- </li></ul><ul><ul><li>Did we do what we said we would do? </li></ul></ul><ul><ul><ul><li>How did we do it? </li></ul></ul></ul><ul><ul><li>Did we achieve what we were seeking to achieve? </li></ul></ul><ul><ul><li>What did we use? </li></ul></ul>9-7-10 LIRG AGM
  11. 11. Project Evaluation <ul><ul><li>What did we get out of it? </li></ul></ul><ul><ul><li>What worked and what didn’t work? </li></ul></ul><ul><ul><ul><li>What could we do differently? </li></ul></ul></ul><ul><ul><ul><li>What can we apply continuously? </li></ul></ul></ul><ul><ul><li>What difference did it make that we did it? </li></ul></ul><ul><ul><li>Who benefited? </li></ul></ul>9-7-10 LIRG AGM
  12. 12. Evidence audit <ul><li>What do you need to know? </li></ul><ul><li>Where is the information? </li></ul><ul><li>Who has the information? </li></ul><ul><li>How will you get it? </li></ul><ul><li>How will you interpret it? </li></ul><ul><li>How will you act on it? </li></ul><ul><li>How will you present it? </li></ul>9-7-10 LIRG AGM
  13. 13. Public library evaluation <ul><li>Derwent Community Library </li></ul><ul><li>MLA study of health and well-being activity in public libraries </li></ul><ul><li>Community engagement achieved by Project LiRA (Libraries in Renewal Areas) in Derby City </li></ul>9-7-10 LIRG AGM
  14. 14. Derwent Community Library <ul><li>Background </li></ul><ul><ul><li>Community library in deprived area </li></ul></ul><ul><ul><li>Newly re-opened, replacing mobile provision </li></ul></ul><ul><ul><li>Shared use building with health & community services </li></ul></ul><ul><li>Purpose </li></ul><ul><ul><li>Requirement of funding </li></ul></ul><ul><ul><li>Develop performance targets </li></ul></ul>9-7-10 LIRG AGM
  15. 15. Method <ul><li>Data analysis </li></ul><ul><li>Interviews </li></ul><ul><ul><li>Senior management </li></ul></ul><ul><ul><li>Branch staff </li></ul></ul><ul><ul><li>User representatives </li></ul></ul>9-7-10 LIRG AGM
  16. 16. Method <ul><li>Surveys </li></ul><ul><ul><li>Adult users </li></ul></ul><ul><ul><li>Child users </li></ul></ul><ul><ul><li>Centre users </li></ul></ul><ul><li>Synthesis and report </li></ul>9-7-10 LIRG AGM
  17. 17. Challenges <ul><li>Relationship with Centre partners </li></ul><ul><li>Attracting use </li></ul><ul><li>Meeting community needs </li></ul>9-7-10 LIRG AGM
  18. 18. Findings <ul><li>Fulfilling a need, but more could be done </li></ul><ul><li>Awareness of the library is good, but could improve. </li></ul><ul><li>User satisfaction is high </li></ul><ul><li>Provides a model for other community projects </li></ul>9-7-10 LIRG AGM
  19. 19. Health and well-being in public libraries <ul><li>Comprehensive overview of current activity </li></ul><ul><li>MLA funded, undertaken in partnership with the Reading Agency, plus expert input from clinical psychologist </li></ul>9-7-10 LIRG AGM
  20. 20. Focus <ul><li>Role of libraries: what activity is going on, and what are its characteristics? </li></ul><ul><li>Partnerships: what partnerships underpin this, and what are the associated benefits and challenges? </li></ul><ul><li>Future potential: what opportunities for development exist and what good practice can be shared? </li></ul>9-7-10 LIRG AGM
  21. 21. Approach <ul><ul><li>Literature review </li></ul></ul><ul><ul><ul><ul><li>Evidence Framework </li></ul></ul></ul></ul><ul><ul><li>Initial stakeholder consultation </li></ul></ul><ul><ul><ul><li>Key opinion leaders (interviews) </li></ul></ul></ul><ul><ul><ul><li>Librarians (survey) </li></ul></ul></ul><ul><ul><ul><ul><li>Activity Map </li></ul></ul></ul></ul><ul><ul><li>3 case studies, involving </li></ul></ul><ul><ul><ul><li>Library practitioners </li></ul></ul></ul><ul><ul><ul><li>Health professionals </li></ul></ul></ul><ul><ul><ul><li>Library users </li></ul></ul></ul>9-7-10 LIRG AGM
  22. 22. Case studies <ul><li>Manchester </li></ul><ul><ul><li>Health Matters - Macmillan Information & Support Service, Beating the Blues </li></ul></ul><ul><li>Stoke-on-Trent </li></ul><ul><ul><li>Better Information, Better Health , and designated Heath Information Officer post </li></ul></ul><ul><li>Northamptonshire </li></ul><ul><ul><li>NHS Health trainers – library hosting and referral, Changing Minds creative bibliotherapy </li></ul></ul>9-7-10 LIRG AGM
  23. 23. Key findings <ul><li>Lots of activity, much in partnership </li></ul><ul><li>Library offer is not well-defined nor publicised </li></ul><ul><ul><li>To partners or public </li></ul></ul><ul><li>Relative invisibility in high level policy and strategy </li></ul><ul><ul><li>But health is high on library agenda </li></ul></ul><ul><li>Partnerships: </li></ul><ul><ul><li>Communication is key </li></ul></ul><ul><ul><li>Tight focus for activity and clear outcomes </li></ul></ul>9-7-10 LIRG AGM
  24. 24. Key findings <ul><li>Libraries provide valuable non-stigmatised community spaces </li></ul><ul><li>Services valued by users </li></ul><ul><li>Evidence of impact is limited </li></ul>9-7-10 LIRG AGM
  25. 25. Project LiRA <ul><li>Libraries in Renewal Areas </li></ul><ul><ul><li>3 new community libraries </li></ul></ul><ul><ul><li>Big Lottery funded </li></ul></ul><ul><li>Aims: </li></ul><ul><ul><li>Developing & delivering services </li></ul></ul><ul><ul><li>Increased access </li></ul></ul><ul><ul><li>New skills </li></ul></ul><ul><ul><li>Heritage </li></ul></ul>9-7-10 LIRG AGM
  26. 26. Project LiRA community engagement <ul><li>Community Engagement Plan </li></ul><ul><ul><li>Community engagement achieved by new libraries </li></ul></ul><ul><ul><li>Before & after opening </li></ul></ul><ul><li>Sustainability and transferability </li></ul><ul><li>Evaluation currently under way </li></ul><ul><ul><li>Timed to opening of libraries </li></ul></ul>9-7-10 LIRG AGM
  27. 27. Approach <ul><li>Documentary evidence </li></ul><ul><li>Interviews with key library staff </li></ul><ul><li>Case studies of each site </li></ul><ul><ul><li>Interviews with </li></ul></ul><ul><ul><ul><li>Branch staff </li></ul></ul></ul><ul><ul><ul><li>Library Panel members </li></ul></ul></ul><ul><ul><ul><li>Library users </li></ul></ul></ul><ul><ul><li>Survey of library users </li></ul></ul><ul><li>Measure against Community Engagement Plan </li></ul>9-7-10 LIRG AGM
  28. 28. Acknowledgements <ul><li>Project funders: </li></ul><ul><ul><li>Derby City Council </li></ul></ul><ul><ul><li>Museums Libraries and Archives Council </li></ul></ul><ul><li>Researchers: </li></ul><ul><ul><li>Helen Greenwood (LISU) </li></ul></ul><ul><ul><li>Debbie Hicks (Reading Agency) </li></ul></ul><ul><ul><li>Valérie Spezi (LISU) </li></ul></ul><ul><ul><li>Mark Summers (LISU) </li></ul></ul>9-7-10 LIRG AGM
  29. 29. Thank you <ul><li>Questions? </li></ul><ul><li>More information: </li></ul><ul><ul><li>www.lboro.ac.uk/departments/dis/lisu </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>9-7-10 LIRG AGM