Use of the SHALL online toolkit for impact assessment: TEWV LIS’s impact study


Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Use of the SHALL online toolkit for impact assessment: TEWV LIS’s impact study

  1. 1. Use of the SHALL online toolkit forimpact assessment:TEWV LIS’s impact studyCatherine EbenezerHealth Libraries North12/05/11
  2. 2. BackgroundImpact on patient care a key measure ofhealth library effectiveness along witheducational benefitsNeed to demonstrate to users, managers andfunding bodies; a requirement of LQAFDifficult!!! Often relies on indirect measuresEarliest efforts in US: Marshall (1992)Very little work in mental health libraries
  3. 3. SHALL impact study• Based on work of Christine Urquhart et al. at theUniversity of Aberystwyth – modified and simplified byNHS South Central• Available at www.libraryservices.nhs.ukUsed three components without modifications:1. Questionnaire survey – can be online2. Schedule for semi-structured interviews3. Questionnaire for requesters of mediated literaturesearches – very similar to main surveyDid not use case study templateRan at TEWV LIS November 2010 – April 2011
  4. 4. Methodology: 1A ‘spare’ Nurse in Medical Education assigned to LIS October –December 2010 – not involved with LIS - methodologically soundTranscribed surveys on to SurveyMonkey (directorate hasProfessional account – can produce reports / analyses)Compiled mailing lists: Registered library members People for whom we had conducted searchesArranged and conducted interviews with people who indicatedtheir willingness to take part:50:50 had used / had not used library when seeking information
  5. 5. Methodology: 2Established required sample size for 95% confidence level and5% confidence interval using standard calculator ( = 357)No questions mandatory – a mistake!Did not attempt to stratify sample of trust staff by professionalgroup, though professional group was identified in resultsMembers and search ‘customers’ circulated via email – with onechaserSurvey featured in trust e-bulletin four times - with varying legend -and on intranet library pagesPrize draw offered!At
  6. 6. Response rates / sample sizes~5000 staff in TEWV:Interview participants: 15Responses to lit. search survey: 18Responses to main survey: 463
  7. 7. Did the information you found influence any of the following?
  8. 8. Did the information you found lead you to CHANGE someaspect of patient care or treatment?
  9. 9. Did the information you found have any impact on your learningor teaching?
  10. 10. Observations from TEWV survey dataResponse rate from nurses is high in absolute terms,though not in relation to their numbers within the trustResponse rates from both training grade and careergrade doctors were low‘No impact’ frequently reported for influence onprofessional activities and patient care (~30%); lessoften on learning or teaching (6%)Advice given to colleagues, service development andplanning the most commonly reported influencesMost commonly reported impact on change of practice= advice given to carers
  11. 11. ‘Please tell us about anything you did differently …’‘The information increased my skills as an educatorand in turn benefited the student’‘The information was used to help decide a patientstreatment’‘I was able to write better-informed reports andassessments’‘I need to be able to research effectively on a regularbasis and library staff have helped me to do this ….’‘I use the library resources to keep myself updated oncurrent best practice’‘Prescribed different medication’‘I became more patient focused with regard toprocesses’
  12. 12. ‘Please tell us about anything you did differently …’‘I was able to discuss his treatment with a very anxiouspatient with a degree of knowledge which gave him theconfidence to persist with a treatment he was findingdifficult’‘Looking at writing a protocol/standard for effectivesupervision practice of medical staff’‘It has started me on the process of networking withother specific professionals working in the same clinicalsetting as me in order to increase my knowledge andshare practice’‘Used information to support behavioural interventionwith family’‘Nothing I’m afraid, colleagues did not take theresearch seriously’
  13. 13. Emerging themes from TEWV interview dataPositive feedback about quality of library service,especially speed of carrying out mediated searches,knowledge of staff – but low staffing levels andrestricted opening hours a problemDelivery of resources to workplaces is greatlyappreciatedMany users previously unaware of range of libraryservices availableLIS needs to raise its profile in clinical areas throughadvertising
  14. 14. ‘Impact’ of the impact study to date ….Presentation to Durham CAMHS – were interested inWeightman and Williamson’s systematic review as wellas our own study – led to interesting discussion as tohow LIS can support CAMHSPresentation to Faculty Development Team – on thebasis of the results asked them to act as ‘evangelists’for the LIS‘Good news’ paper for EMT – to prefacedocumentation for clinical librarian projectFull report not yet written!
  15. 15. Catherine EbenezerLibrary and Information Services Managercatherine.ebenezer@tewv.nhs.uk01642 83811207920