'Workshop on Smart Searching: Search Filters and Expert Topic Searches', by Sarah Hayman and Raechel Damarell, Flinders University
•Download as PPTX, PDF•
1 like•1,066 views
Presented at the 8th International Evidence Based Library and Information Practice Conference, Brisbane, July 2015
Recommended
More Related Content
What's hot
What's hot (19)
Similar to 'Workshop on Smart Searching: Search Filters and Expert Topic Searches', by Sarah Hayman and Raechel Damarell, Flinders University
Similar to 'Workshop on Smart Searching: Search Filters and Expert Topic Searches', by Sarah Hayman and Raechel Damarell, Flinders University (20)
More from CareSearch palliative care knowledge network
More from CareSearch palliative care knowledge network (20)
Recently uploaded
Recently uploaded (20)
'Workshop on Smart Searching: Search Filters and Expert Topic Searches', by Sarah Hayman and Raechel Damarell, Flinders University
- 1. Workshop Smart Searching: Search Filters and Expert Topic Searches Sarah Hayman and Raechel Damarell 8th International Evidence Based Library and Information Practice Conference Brisbane, July 2015
- 2. Smart Searching • Introduction to search filters: what are they, where to find them, how to use them • Development process for a search filter • Using the Smart Searching modules • Tips, tricks and tools
- 3. Searching Well Why important? • Need for evidence • Impact of missing evidence • Volume of information • Complexity of sources
- 4. A search filter … • is an experimentally developed and validated search strategy of known performance effectiveness • is designed for a particular bibliographic database • can be methodology- or subject-based • may also be called a “hedge” • may be sensitive or specific • can be expressed as a URL and embedded in a web page for quick reliable access to evidence
- 5. Sensitivity and precision • Sensitivity (recall) = the proportion of relevant articles retrieved – 100% sensitivity occurs when all relevant citations in a dataset are retrieved • Precision = the proportion of relevant articles retrieved as a proportion of all articles retrieved – 100% precision occurs when all citations retrieved are relevant
- 6. Specificity • Specificity = the proportion of irrelevant citations not retrieved – 100% specificity occurs when all irrelevant citations in a dataset are all excluded from the search results
- 7. Balancing these metrics • Generally speaking, search filters will attempt to maximise precision without jeopardising sensitivity • Systematic review searches aim to maximise sensitivity without jeopardising precision • Often different versions of same search filter on offer (e.g. Clinical Queries)
- 8. Clinical Queries Therapy Search Filter Filter version Sensitivity Specificity Precision Therapy (sensitive/broad) 99% 70% 10% Therapy (specific/narrow) 93% 97% 54% PubMed Help [Internet]. Bethesda (MD): National Center for Biotechnology Information (US); 2005-. PubMed Help. [Updated 2015 Jun 16]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK3827/
- 9. Using Search Filters • Nature and purpose of the search filter • Not designed for systematic review searching • Use terms in search filter a starting point for your own searches • Critically appraise the search filter • Make search filters available to your users as useful tools for accessing evidence reliably – Embed in webpages/libguides – Autoalerts
- 10. Search Filters • CareSearch • Flinders Filters • Some search filters worldwide – ISSG Search Filters Resource – BMJ Clinical Evidence – Canadian Agency for Drugs and Technologies in Health (CADTH) – HL WIKI International – McMaster University. Health Information Research Unit – Scottish Intercollegiate Guideline Network (SIGN)
- 11. Example: the Heart Failure Search Filter As a URL: http://www.ncbi.nlm.nih.gov/pubmed?term=((heart+failure[tw]+OR+ventricular+dysfu nction,+left[mh:noexp]+OR+cardiomyopathy[tw]+OR+left+ventricular+ejection+fraction [tw])+AND+Medline[sb])+OR+((heart+failure[tw]+OR+left+ventricular+dysfunction[tw]+ OR+cardiomyopathy[tw]+OR+left+ventricular+ejection+fraction[tw]+OR+cardiac+resync hronization[tw]+OR+LV+dysfunction[tw]+OR+left+ventricular+systolic+dysfunction[tw]+ OR+left+ventricular+diastolic+dysfunction[tw]+OR+cardiac+failure[tw])+NOT+medline[s b])+AND+english[la] As a PubMed query:
- 13. Workshop activity 1 Workshop website at https://sites.google.com/site/eblip8 searchingworkshop/home
- 14. Smart Searching Expert Advisory Group (EAG) Subject experts Gold Standard Set Sample set Term identification Term identification Validation Testing
- 15. The Smart Searching Modules https://sites.google.com/site/smartsearchinglogical/home
- 16. Acknowledgements Thank you to: Our colleagues at CareSearch and Flinders Filters (Mikaela Lawrence, Yasmine Shaheem, Jennifer Tieman) Health Libraries Australia and Medical Director (formerly HCN) for the Health Informatics Innovation Award 2012 that supported the development of Smart Searching Raechel Damarell and Sarah Hayman, Flinders University raechel.damarell@flinders.edu.au sarah.hayman@flinders.edu.au
Editor's Notes
- Want to be informal and relaxed – we are all colleagues – we are not necessarily any more expert than you in searching – chance for us to share our ideas and experience with you and get some feedback – and hear about your ideas and experience and searching tips. Pleas interrupt with any questions – don’t sit there wondering! Will aim to make it interactive, give you a chance to do some hands on exercises.
- Today this is what we aim to cover in the workshop. We want you to ask questions and we will try to work through all the aspects and demonstrate examples of what we mean. There will be a chance to interact with some data and play with the techniques. Will also give you links to useful tools and websites. We are also always happy to be contacted with any questions – contact details will be at the end of the presentation and on the workshop website.
- Don’t need to convince this audience of any of this – you all are well aware of both the growing importance and challenge of searching effectively. Maybe you can suggest other ideas, examples here? The importance (in all fields) of finding and using evidence is growing rapidly, with increased recognition that decisions should be based on sound evidence. Also important to remember the potential impact of missing a piece of evidence – inadequate searching can result ultimately in adverse outcomes for the patient – we should all always remember this. Key to finding this evidence is effective searching. Alongside this imperative, the searching context is becoming more complex. The number of articles indexed is enormous and increasing. In the medical field, PubMed contains over 24 million citations with over 1 million entered in 2014. Effective searching requires an understanding of database mechanisms and the terminology (including associated thesauri) of each subject. Searchers need an understanding of the requirements of the end user: what is considered relevant and what are the levels of evidence?
- So what is a search filter? Not pubmed sidebar limits Search strategy – set of terms – that has been tested and its performance measured. The process is transparent and rigorous. It is a scientific approach to searching where the process is documented and defensible. Evidence based approach. Important to understand that a search filter is built to perform in a particular database. It may be translated for use in another database but it will not necessarily have the same retrieval effectiveness in a different database. Search filters can be methodology based –m that is they look for articles using a particular methodology, such as systematic reviews. The PubMed Clinical Queries are search filters looking for particular methodologies. The can also be topic based, as are the ones we have created at CareSearch and FF. Our filters are on topics such as bereavement, palliative care or primary health care. Search filters once created can be expressed as a URL if the database allows that. Our PubMed filters can be created this way and delivered as links on a web page to our users so they can simply click on a link to get real time search results. This is a very useful tool that you can actually use for your users too. Show PubMed searches eg bereavement Just to note you will sometimes see the term Hedge used for search filter Important to remember a search filter is tweaked to perform in a certain way. They may be highly sensitive (that is retrieve a very high percentage of relevant references) or highly specific, that is, not retrieve a high percentage of irrelevant references. Dialling up the sensitivity in a search will tend to suppress specificity and vice versa. Both types are important in different contexts and you will sometimes see two versions published of the same filter – one sensitive the other specific. See e.g. the Haynes Nephrology filters. For a comprehensive search such as systematic review you would want to use the sensitive filer; for a quick search for a busy clinician the specific filter is likely to be more useful.
- Perfect sensitivity = 100 relevant citations in a database and you retrieve all 100. Precision = Possible to have very high precision yet very low sensitivity. Again there may be 100 relevant citations in a database. Say you retrieve only 10 citations and they are all relevant – you have 100% precision. Sensitivity however is only 10/100 = 10%. Opposite is true – you would have 100% sensitivity if you retrieve all 100 relevant citations in the database. However, if you retrieve 1000 citations in order to capture the 100 relevant ones though, precision only 10% Filter methodology makes it possible to measure sensitivity, but real world searching can’t. Don’t know the number of relevant citations in the database so denominator is an unknown. You can, however, measure precision of your own searches. So searching can only ever be a blind attempt to maximise sensitivity while enhancing precision.
- Just a reminder again that search filters may not be suitable for a systematic review search. In any case, when doing sys rev searching you would alwys look in more than one place. Note the term search filter is widely used and does not always mean the specific search filter that we refer to in the bibliographic searching context. PubMed call the limits on the sidebar filters – some are fully developed search filters, others are simply date limits. If you put the term search filter into Google many of the results coming back will be about search filter work in IT. Nature and purpose: search filter development is guided by an interest group. In the case of our search filters, we use an expert advisory group containing clinicians and researchers. Their needs and/or understanding of the concept shape the filter. It is useful to try to understand the purpose and context of a filter in order to know whether it is what you need to use. What is it designed to retrieve exactly? Is that what you are looking for? You can critically appraise search filters. There are tools available and often there will be a published paper establishing the existence of the filter and documenting the methodology used to develop it and the validation that was undertaken. McMaster: The purposes of the search filters are: 1. to enable health care providers to do their own clinical searches effectively and efficiently; 2. to help reviewers of published evidence concerning health care problems to retrieve all relevant citations; 3. to provide resources for librarians to help clinicians to construct their own searches; and 4. to provide input to the database producers about their indexing processes and the organization of their databases. A simple way to use search filter is to look at it and at least use the terms in it as a starting point for your search. You can always supplement the search filter results with your own. You can make search filters available to your users - link to them and even create your own topic searches using search filters that you can then publish on a web site or libguide.
- Useful tools in the searcher’s armoury are search filters. At CareSearch and its associated project Flinders Filters we have developed now a number of topical or subject-based search filters. These are designed to search the medical literature – indeed all so far have been created to search Medline and PubMed, on particular topics. Looking quickly at our two websites, we can see the topics listed: go to the links and show the names. These filters are freely available for all to use. I’ve also shown here some links to international websites with search filters and information about them. If you are interested in search filters and want to learn more, it is well worth going to these sites and exploring. There are many different filters and different types of search filter.
- Here is an example of what a search filter looks like. This is the Heart Failure Search Filter, created by CareSearch. It is shown as a URL and also as a PubMed search string which is a little easier to read. It looks in the indexed part of PubMed using the equivalent of the Medline search filter but it also uses text words (natural language terms) in the second part of the filter which looks in the non indexed part. This second part is also tested for relevance when we develop the PubMed translation.
- This is an outline of our search filter development Process. Talk briefly though each step. It is a major piece of work which takes several months. Several librarians work on these projects and we found we were often talking to each other about how the techniques we use in this process have enhanced our own general literature searching and could be more widely applied. When we received the HLA/HCN Health Informatics innovation award in 2012 this was an opportunity to develop an online module exploring these ideas and suggesting adaptations of some of this filter development methodology. It is not at all the full methodology we use to develop a search filter but takes some elements and suggests how they could be applied in general searching – to enhance searching and provide evidence of searches’ effectiveness. Want to make the important point here too that we know many librarians are already expert searchers and we suspect you may already be doing many of these things we suggest in your own searching – these are some tips and tricks we think may perhaps systematise searching in a useful way, or be handy for training new searchers – and it may be new to consider how you can quantify your searching effectiveness. The four points we chose to focus on in the search filter dev process shown here are: the EAG, the GSS, Term identification and Validation.
- We have adapted and translated four key elements of the Search Filter development methodology four points as follows: The EAG becomes Subject Experts in our training modules. The Gold Standard Set becomes Sample Set in our training modules. Term identification we felt was self explanatory and remains the same in our training modules. Validation we have simply called Testing in our training modules.
- Now go to the modules and work through the examples there.