Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons from the Drug-Drug Interaction Extraction Task


Published on

Authors: Isabel Segura-Bedmar, Paloma Martínez, Daniel Sánchez-Cisneros.
DDI Extraction 2011: Drug-Drug Interaction Extraction 2011, Huelva, España (September 5, 2011)
Lessons from the Drug-Drug Interaction Extraction Task

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Lessons from the Drug-Drug Interaction Extraction Task

  1. 1. DDIExtraction 2011 Lessons from the Drug-Drug Interaction Extraction Task Huelva, September, 7th Isabel Segura-Bedmar, Paloma Martinez, Daniel Sánchez- Cisneros, LABDA, UC3M
  2. 2. Drug-Drug Interactions (DDIs)  A DDI occurs when one drug influences the level or activity of another drug.  DDIs are a serious problem for patient safety.  Overwhelming amount of information available on DDIs: databases, journals, technical reports, books, etc.  Information Extraction (IE) can provide an interesting way of reducing the time spent by health care professionals on reviewing the literature.
  3. 3. DDIExtraction Task  Automatic extraction of drug-drug interactions from text  Main goals:  To produce useful resources for training and testing.  To learn which approaches are successful and practical.  To encourage in developing useful tools to extract DDIs from texts.
  4. 4. DDIExtraction Task
  5. 5. Summary of Approaches • Main approaches at the top level – Markov – Support Vector Machine (SVM) • Typically with a small window around the word of interest – Rules: typically manually generated? • Also may have pre- or post-processing in addition to the main approach • Lots of different features used – Some features need entire other sub-systems to obtain a value • E.g., a part-of-speech (POS) tagger • The other sub-system may use an approach different from the main one
  6. 6. Summary of Results: Comments • Like many similar tasks, task 1A has its share of unique aspects – The 1-2 teams that did not really train against the training data did not do as well • Results for open and closed submissions are typically fairly close • At the higher scores • Between the submissions from the same user
  7. 7. DDIExtraction 2011 Closing remarks Huelva, September, 7th Isabel Segura-Bedmar, Paloma Martinez, Daniel Sánchez- Cisneros, LABDA, UC3M
  8. 8. DDIExtraction 2011 Task  Pilot task.  Several limitations:  Drug names were automatically annotated.  Annotation process performed by one unique annotator.  Many annotation errors.
  9. 9. DDIExtraction Task in SEMEVAL 2013
  10. 10. DDIExtraction Task in SEMEVAL 2013  New train and test datasets: unified format.  Identify drug-drug interaction pairs from full text articles (HTML, PDF).  Drugs and drug-drug interactions manually annotated by two different pharmacists.  Calculate Inter-annotator agreement (IAA).
  11. 11. DDIExtraction Task in SEMEVAL 2013  September 7, 2011 Call for participation  April 10, 2012 Full Training Data available for participants  January 1, 2013 Start of evaluation period  February 1, 2013 End of Evaluation Period  March 1, 2013 Paper submission deadline  Summer 2013 Workshop co-located with ACL or NAACL
  12. 12. Acknowledgments  DrugBank for providing the training and test data collections.  Manuel Maña for collaboration in organizing the DDIExtraction task.  Thanks to the PC members for their invaluable help.  Thanks to the participating teams for their effort in developing the participating systems and improving the quality of the datasets.
  13. 13. Thanks!!! See you on SEMEVAL 2013!!!