0
Signal Detection ofAdverse Medical DeviceEvents in theFDA MAUDE DatabaseEric Brinsfield, MSPresenter & Research Collaborat...
Disclosure Statement Both presenters are employees of SAS Institute We have no conflicts of interest DisclaimerThe views...
Study ObjectivesDetermine if text mining can be used to:   Detect signals of adverse events in    spontaneous reporting d...
Definitions Safety Signal   “A report or reports of an event with an unknown    causal relationship to treatment that is...
Methods Data Source = MAUDE     FDA Spontaneous Reporting System Database for Med. Devices      MAUDE - Manufacturer and...
MAUDE (2001-2008)Spontaneous “safety” reports on medical devicesStrengths:   Only surveillance system which covers device...
Characteristics of Spontaneous Report Suitable for hypotheses generation; not for confirmation or  rates computation Exh...
Study Design: Case-Series                  Source Population     All MAUDE reports received between 2006 – 2010           ...
Analysis Steps1. Attempt analysis using text mining alone     Ignore device     Evaluate if general text mining provides...
Text Mining Process Parse terms to create documents-terms frequency matrix Use singular value decomposition (SVD) to mea...
Sample Narrative - InjuryExample of Manufacturer Report - Injury the product labeling for a p154 states that this product...
Concept Linking and Exploration                                                                           12              ...
Text mining clusters over all reports                                                                            13       ...
Text Mining Over All Reports Results are interesting But inconclusive Lose track of the device But may detect new tren...
Disproportionality Results for   Carotid Stent (NIM)                                                                      ...
Target for Text Mining Evaluation NIM showed high proportion of “Other”    Based on relative percentage of reports    B...
NText Mining Clusters for NIM+percutaneous +unk dissection performance +unstable repaired                                3...
18    Company Confidential - For Internal Use OnlyCopyright © 2010, SAS Institute Inc. All rights reserved.
Manual Review of Text for “Other” Most were not adverse events that persisted Some seemed like “FYI” reports. Two inclu...
Conclusion Text mining shows promise for recognizing  primary words and patterns Hard to form hypotheses from bulk text ...
Next StepsNeed further analysis that includes: Large target group for further triage    “Other” was too small in this ca...
Final Thoughts“What the future portends is more and moreinformation — Everests of it. There won’t beanything we won’t know...
Thank YouContacts: Eric Brinsfield  eric.brinsfield@sas.com David Olaleye  david.olaleye@sas.com                       C...
Upcoming SlideShare
Loading in...5
×

Text Mining the MAUDE Database

783

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
783
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Text Mining the MAUDE Database"

  1. 1. Signal Detection ofAdverse Medical DeviceEvents in theFDA MAUDE DatabaseEric Brinsfield, MSPresenter & Research CollaboratorDavid Olaleye, MSCE, PhDAuthor & Primary Research StatisticianSAS Institute Inc.Cary, NC Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  2. 2. Disclosure Statement Both presenters are employees of SAS Institute We have no conflicts of interest DisclaimerThe views and opinions expressed in the followingPowerPoint slides are those of the individual presentersand should not be attributed to ISPE or to SAS Institute. 2 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  3. 3. Study ObjectivesDetermine if text mining can be used to:  Detect signals of adverse events in spontaneous reporting data  Better understand or triage signals generated by traditional disproportionality methods Phase 1:  Evaluate unsupervised text mining  Using FDA MAUDE database  Focused on stents 3 Company Confidential - For Internal Use Only 3 Copyright © 2010, SAS Institute Inc. All rights reserved.
  4. 4. Definitions Safety Signal  “A report or reports of an event with an unknown causal relationship to treatment that is recognized as worthy of further exploration and continued surveillance.” » Council for International Organizations of Medical Sciences (CIOMS) “Recognition” is often the results of  Analytical or automatic signal detection methods that look for unexpected patterns in data sources such as: » Spontaneously reported data » Observational healthcare data » Insurance claims data 4 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  5. 5. Methods Data Source = MAUDE  FDA Spontaneous Reporting System Database for Med. Devices MAUDE - Manufacturer and User Facility Device Experience  Contains the narrative entered by the reporter Target Devices  endovascular graft system and coronary stent devices  devices classified as a stent in the “product_category_code”: » MAF, MIH, NIN, NIO, NIP, NIM, and NIQ » http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/classif ication.cfm?ID=896 5 Company Confidential - For Internal Use Only 5 Copyright © 2010, SAS Institute Inc. All rights reserved.
  6. 6. MAUDE (2001-2008)Spontaneous “safety” reports on medical devicesStrengths:  Only surveillance system which covers devices marketed in the entire US  Largest number of case reports on adverse outcomes and malfunctions for medical devices  Provides opportunity to detect signals of new, rare and unusual adverse clinical outcomes » Usually warrants further investigation  Includes narrative description of events » Better than AERS which does not include narrative 6 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  7. 7. Characteristics of Spontaneous Report Suitable for hypotheses generation; not for confirmation or rates computation Exhibit under-reporting and other reporting biases; lack of control group, etc. Data quality, ascertainment, accuracy and completeness of information are usually poor Includes events and incidents not causally related to medical device exposure Does not distinguish between label versus off-label uses of approved products Contains minimal or no patient history and other potential causal factors Do not provide estimates of exposure (worse in drugs) 7 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  8. 8. Study Design: Case-Series Source Population All MAUDE reports received between 2006 – 2010 (N=35954)Study Device Cohort Study EventsMAUDE reports for stents • Death (D)with product codes: • Injury (I) MAF, MIH, NIN, NIO, • Malfunctions (M) NIP, NIM, and NIQ • Other (O) Device-Adverse Outcomes Pairs (N=28) 8 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  9. 9. Analysis Steps1. Attempt analysis using text mining alone  Ignore device  Evaluate if general text mining provides any insights2. Perform standard disproportionality analysis on structured data  PRR, EBGM, Adj. Residual3. Identify device-AE pairs that have:  High scores  Especially in the “Other” classification4. Investigate the device with text mining  Include all outcome classifications 9 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  10. 10. Text Mining Process Parse terms to create documents-terms frequency matrix Use singular value decomposition (SVD) to measure association and perform hierarchical clustering Use entropy method to cluster SVDs for documents classification 10 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  11. 11. Sample Narrative - InjuryExample of Manufacturer Report - Injury the product labeling for a p154 states that this product is indicated for use in pts with obstruction of major biliary ducts. the product labeling also states that the stent may be increased post-placement by expanding with a larger diameter balloon. the following was obtained through conversation with the user facility on 1/15/98. after deciding that a ptca procedure in a renal artery did not yield adequate results, the md attempted to place a medium biliary stent in the artery. the physician reported to co. that he had difficulty visualizing the stent and that it was difficult to place. he deployed the stent, but was not satisfied with the outcome. in response, he decided to place a second stent inside of the first. however, the stents interlocked and the physician decided to remove both stents, he was able to withdraw the stent up to sheath tip in the femoral artery, but needed a vascular surgeon to completely remove them. info regarding the type of removal procedure has not been provided to co. the physician further stated that he believes that the first stent had not fully opened. it was mounted on a meditech glidex balloon. 11 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  12. 12. Concept Linking and Exploration 12 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  13. 13. Text mining clusters over all reports 13 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  14. 14. Text Mining Over All Reports Results are interesting But inconclusive Lose track of the device But may detect new trends for all stent devices in studyCannot really do comparisons due to lack of denominator.(Same problems as always with spontaneous reports.)Next, run disproportionality to narrow the focus… 14 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  15. 15. Disproportionality Results for Carotid Stent (NIM) Adjusted MGPSAdverse Event Freq Residual (EBGM)Death 276 0.8 0.86728Injury 2043 1.1 1.130264Malfuntion 271 0.4 0.424803Other 24 1.6 3.648191 * Adjusted Residual: Flagged at values over 1.5 EBGM: Flagged at values over 2.0 15 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  16. 16. Target for Text Mining Evaluation NIM showed high proportion of “Other”  Based on relative percentage of reports  Based on signal scoring algorithms  All methods suggested a flag  Although only 24 cases, the method could show promise Run text mining against all NIM reports  Include all outcomes to fully understand reports  Look for possible explanations or hypotheses 16 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  17. 17. NText Mining Clusters for NIM+percutaneous +unk dissection performance +unstable repaired 376+malfunction device remains implantedreactions collapsing +premature deployment cracks +product quality 41issue replaced performance collapsedmalfunction fractures +fracture +inaccurate delivery fractured 846malfunctions drift +premature deployment+normal dissection +unk no information +na collapsing unk fractured 69device issue broke no known device problem +break broken +shaft 227break reaction reactionsabnormal +fracture +continuous +bent cracks unk +break fractured 65filter +na difficult to advance breaks reaction +bent +malfunction 150replaceddevice remains implanted collapsed no flow performance +crack 44+break +collapse filterfractured repair +crack breaks +collapse +product quality issue no 31flow reaction 17 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  18. 18. 18 Company Confidential - For Internal Use OnlyCopyright © 2010, SAS Institute Inc. All rights reserved.
  19. 19. Manual Review of Text for “Other” Most were not adverse events that persisted Some seemed like “FYI” reports. Two included notification of a formal study Most patients still had the stent in place (assumed) Some cases of installation problems:  Potential installer error  Most did not involve an adverse event 19 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  20. 20. Conclusion Text mining shows promise for recognizing primary words and patterns Hard to form hypotheses from bulk text mining on spontaneous database Combination with disproportionality analysis creates signals that can be further analyzed with text mining Terms in the “Other” category overlap with other categories 20 Company Confidential - For Internal Use Only 20 Copyright © 2010, SAS Institute Inc. All rights reserved.
  21. 21. Next StepsNeed further analysis that includes: Large target group for further triage  “Other” was too small in this case Preferred term matching and encoding  To clean up fuzziness and reduce clusters Content categorization  Look for more structure and combine with ontologies Sentiment analysis  Determine if overall sentiment was good or bad 21 Company Confidential - For Internal Use Only 21 Copyright © 2010, SAS Institute Inc. All rights reserved.
  22. 22. Final Thoughts“What the future portends is more and moreinformation — Everests of it. There won’t beanything we won’t know. But there will be noone thinking about it.” From: New York Times - August 13, 2011 The Elusive Big Idea By NEAL GABLER Neal Gabler is a senior fellow at the Annenberg Norman Lear Center at the University of Southern California and the author of “Walt Disney: The Triumph of the American Imagination.”We need to help make time for thinking. 22 Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  23. 23. Thank YouContacts: Eric Brinsfield  eric.brinsfield@sas.com David Olaleye  david.olaleye@sas.com Company Confidential - For Internal Use Only Copyright © 2010, SAS Institute Inc. All rights reserved.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×