Machine Learning Assisted Citation Screening for Systematic Reviews
•Download as PPTX, PDF•
0 likes•94 views
Evidence-based practice is highly dependent upon up-to-date systematic reviews (SR) for decision making. However, conducting and updating systematic reviews, especially the citation screening for identification of relevant studies, requires much human work and is therefore expensive. Automating citation screening using machine learning (ML) based approaches can reduce cost and labor. Machine learning has been applied to automate citation screening but not for the SRs with very narrow research questions. This paper reports the results and observations for an ongoing research that aims to automate citation screening for SRs with narrow research questions using machine learning. The research also sheds light on the problem of class imbalance and class overlap on the performance of ML classifiers when applied to SRs with narrow research questions.
Recommended
More Related Content
What's hot
What's hot (18)
Similar to Machine Learning Assisted Citation Screening for Systematic Reviews
Similar to Machine Learning Assisted Citation Screening for Systematic Reviews (20)
More from Anjani Dhrangadhariya
More from Anjani Dhrangadhariya (6)
Recently uploaded
Recently uploaded (20)
Machine Learning Assisted Citation Screening for Systematic Reviews
- 1. Machine learning assisted citation screening for Systematic Reviews Anjani K. Dhrangadhariya et al. MedGIFT group University of Applied Sciences Western Switzerland (HES-SO) Slides adapted from MIE2020 conference 1
- 2. Background • Physicians moving away from anecdotal evidence for treatment • Shift towards Evidence-based Medicine (EBM) • Combining best available evidence with clinical judgement 2
- 3. Background • Evidence aggregation? • Systematic reviews • Example questions: 1. Will acupuncture combined with the drug modafinil reduce fatigue in adult cancer patients? 2. Could Botox injections reduce abnormal muscle function in children aged 8- 14 years? 3. Will absorbable sutures be better than metal sutures during abdominoplasty for older patients? 3
- 4. Background Formulate clinical question Search & Collect scientific studies Manually screen the studies Relevant studies Irrelevant studies • Google scholar • PubMed • EMBASE • CINAHL • PsychInfo • Cochrane CENTRAL • clinicaltrials.gov • Grey literature Include Exclude 4 Further analysis
- 5. Motivation Pros Cons Systematic process Time consuming • 12 – 24 months Evidence from multiple sources Laboursome • At least two doctors need to manually screen the studies • ~90,000 studies each Reliable results High cost • Extra salary for doctors! Single question! 5
- 6. • Automatic screening could ease doctors’ burden. Motivation and Objective Manually screen the studies Relevant studies Irrelevant studies Include Exclude 6 Manually
- 7. Data source • Dataset from Hilfiker et al. (2017) • 31’279 titles and abstracts • Manually screened into • Two classes (labels) • 4’066 – Include • 27’213 – Exclude PubMed Central – Open Access: 2.09 Mi. titles and abstracts to train word embeddings. 1 2 Generate word embeddings Explore screening automation PoC I E < 7
- 8. Methodology 1. Generate word embeddings • Input: 2.09 Mi titles and abstracts from PMC • Output: word embeddings • Further text preprocessing and hyperparameter information in the paper. Text preprocessing Phrase generation: word2phrase Word tokenization Unigrams bigrams word2vec fastText word2vec embeddings fastText embeddings 1 8
- 9. Methodology 1. Screening automation • Input: 31’279 titles and abstracts • Train-test split: 80-20% Deduplication Text preprocessing No oversampling Feature extraction Classifier training and test Random oversampling 2 9 Input corpus: titles and abstracts I E < • Logistic Regression • Support Vector Machines • K-Nearest Neighbor • Decision Trees • Random Forests • Convolutional Neural Networks
- 11. Discussion 11 E I Cosine similarity (centroid): 0.985814 Cosine similarity (centroid): 0.985824
- 12. Conclusion • One of the first to explore such narrow topic (physiotherapy) citation screening topic • Using domain-specific word embeddings for citation screening • Exploring the topic of • Class imbalance • Class overlap • This problem disguises as classification problem but it is not. 12 E I I E <
- 13. Further reading • Dhrangadhariya A, Hilfiker R, Schaer R, Müller H. “Machine Learning Assisted Citation Screening for Systematic Reviews.” Stud Health Technol Inform. 2020;270:302-306. doi:10.3233/SHTI200171 • Summary of papers in BioNLP workshop from ACL2020: https://medium.com/@recurrent.pi/summarizing-papers-from- bionlp-workshop-in-acl2020-a6ba3d937705 13
- 14. Thank you for your attention Email anjani.dhrangadhariya@hevs.ch LinkedIn https://www.linkedin.com/in/anjani-dhrangadhariya/ Twitter https://twitter.com/AsciiRandom Medium https://medium.com/@recurrent.pi 14