Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Identifying Safety Signals by Data Mining the FDA Adverse Event Reporting System with Empirica Signal


Published on

Ever since the European Union (EU) introduced new legislation that requires life sciences companies to proactively detect, prioritize, and evaluate safety signals, there has been an increased …

Ever since the European Union (EU) introduced new legislation that requires life sciences companies to proactively detect, prioritize, and evaluate safety signals, there has been an increased interest, not only from sponsors and CROs in the EU, but globally, in pharmacovigilance systems that can assist with the signal management process.

Perficient's Chris Wocosky, an expert in signal detection and management, shows how your organization can use Empirica Signal, Oracle's state-of-the-art signal detection system to data mine the existing FDA Adverse Event Reporting System (FAERS) to determine safety signals. This presentation and demonstration willhelp you bettter understand how this solution can be used in daily pharmacovigilance activities.

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Identifying Safety Signals By Data Mining the FDA Adverse Event Reporting System with Empirica Signal Chris Wocosky, Lead Business Consultant Life Sciences Business Unit
  • 2. Perficient is a leading information technology consulting firm serving clients throughout North America and Europe. We help clients implement business-driven technology solutions that integrate business processes, improve worker productivity, increase customer loyalty and create a more agile enterprise to better respond to new business opportunities. About Perficient
  • 3. • Founded in 1997 • Public, NASDAQ: PRFT • 2013 revenue ~$373 million • Major market locations throughout North America Atlanta, Boston, Charlotte, Chicago, Cincinnati, Columbus, Dallas, Denver, Detroit, Fairfax, Houston, Indianapolis, Los Angeles, Minneapolis, New Orleans, New York City, Northern California, Philadelphia, Southern California, St. Louis, Toronto and Washington, D.C. • Global delivery centers in China, Europe and India • >2,100 colleagues • Dedicated solution practices • ~90% repeat business rate • Alliance partnerships with major technology vendors • Multiple vendor/industry technology and growth awards Perficient Profile
  • 4. Oracle Products • Argus Safety Suite • Argus Safety Japan • Affiliate • Interchange • Reconciliation • Dossier • Insight • AERS • TMS • Empirica Signal • Empirica Topics Our Solutions Expertise Life Sciences Business Unit Safety & Pharmacovigilance Provides Consulting, Implementation, Integration, Upgrade, Migration and Hosting Services for the most widely used and highly sought after drug safety, pharmacovigilance, risk, and signal management systems in the world.
  • 5. Introduction Industry Experience: 17 years • Medical Device Manufacturing – Edwards Lifesciences (formerly Edwards Laboratories) Abbott Vascular (formerly Guidant), J&J Biosense Webster • Cardiovascular and Electrophysiology – Tissue & Mechanical Heart Valves, Coronary Stents and Ablation Catheters • Quality/Reliability Engineering, Quality Assurance, Regulatory Compliance and Complaint Management Software Experience: 8 years • Software Application Sales – ArisGlobal, Phase Forward, Oracle, BioPharm Systems (now Perficient) • Safety & Pharmacovigilance Products – Register, inforMed, ARISg; – ClinTrace, Empirica Signal, Study and Topics – Siebel AECM, Argus Safety Suite • Pre-Sales Consulting, Implementation Consulting, Training Christina Wocosky Lead Business Consultant Safety and Pharmacovigilance Life Sciences Business Unit o: 951-676-9501 m: 951-505-7011
  • 6. Agenda • Background of Empirica Signal • Why Empirica Signal? • Product Overview • Q&A
  • 7. Background • Developed 2003-2005 as a (CRADA) Cooperative Research and Development Agreement between FDA and Lincoln Technologies, Inc. lead by Bill DuMouchel and Ana Szarfman. • Objective: User friendly / web based tool for FDA • WebVDME – Web Visual Data Mining Environment • Conclusion: Tool assists in identifying unusual patterns – BUT – the patterns need interpretation! • 2005 - Phase Forward acquired LT – rebranded as Empirica Signal • 2010 - Oracle acquired Phase Forward • In use at FDA today
  • 8. Regulatory Landscape • Increased focus on technology to save time and money • Companies are expected to do more with tighter budgets • Global requirements growing – still not harmonized • An increased focus on safety related issues
  • 9. Drugs Withdrawn Last 5 years* Withdrawn Drug name Remarks Country(s) 2013 Tetrazepam Serious cutaneous reactions. EU 2011 Drotrecogin alfa (Xigris) Lack of efficacy as shown by PROWESS-SHOCK study WorldWide 2010 Propoxyphene (Darvocet/Darvon) Increased risk of heart attacks and stroke. WorldWide 2010 Gemtuzumab ozogamicin (Mylotargz) No improvement in clinical benefit; risk for death. US 2010 Rosiglitazone (Avandia) Risk of heart attacks and death. This drug continues to be available in the U.S. EU 2010 Sibutramine (Reductil/Meridia) Increased risk of heart attack and stroke. Australia, Canada, China, the EU, Hong Kong, India, Mexico, New Zealand,the Philippines, Thailand,the UK, and the US 2010 Sitaxentan Hepatotoxicity Germany 2009 Efalizumab (Raptiva) Increased risk of progressive multifocal leukoencephalopathy Germany *See References
  • 10. Why These Drugs Were Withdrawn • Risks to the patients, serious side effects • Unexpected AE’s that were not detected during Phase III Clinical Trials • Only became apparent from post-market surveillance data from the wider patient community • Not limited to drugs that were ever approved by the FDA • Some were approved in markets other than the US
  • 11. Empirica Signal Solution Area • Growing government and public concern about drug safety has increased the importance of pharmacovigilance – Traditional methods such as individual case review no longer suffice due to increased data volume and complexity – New strategies to proactively identify and expeditiously manage emerging safety risks are needed • In this highly competitive and risk-averse environment, it is essential for regulatory authorities as well as pharmaceutical companies to establish pharmacovigilance programs – Need to capitalize on multiple data sources using advanced tools for developing a thorough and well understood safety profile
  • 12. Process Proposed by CIOMS VIII Signal Prioritization Signal Detection Signal Evaluation
  • 13. GVP Module IX The signal management process shall cover all steps from detecting signals (signal detection), through their validation and confirmation, analysis, prioritization and assessment to recommending action, as well as the tracking of the steps taken and of any recommendations made.
  • 14. The GVP Module IX Process Signal Detection Signal Validation Exchange of Information Empirica Signal Empirica Topics Signal Analysis and Prioritization Signal Assessment Recommendation for Action
  • 15. Why Empirica Signal/Topics? • The same tool developed and in use by FDA, as well as the MHRA • Early detection of new potential signals, or changes in signal profiles over time • Track and document every step you take from the first moment a potential signal is identified, monitoring, verification, decisions and all actions taken • Provide regulators with a .pdf with all the necessary information to back up decisions If you didn’t document it, it didn’t happen!
  • 16. Signal Detection and Management Signal Detection • Monitoring of “Drug Profiles” with visualization tools • Quick and easy drill down to the data under the charts and graphs • Create your own data mining runs to further refine your analysis • Compare “like” products side by side with the same charts and graphs Signal Management • Integrate quantitative signal detection techniques into routine safety reviews • Assess the latest information on all marketed products and how “signals” evolve as data accumulates over time • Store public and private comments and blanket comments for multiple drug-event combinations • Store findings with all other data sources
  • 17. Empirica Signal Benefits • Early detection of potential safety signals for marketed drugs • Multiple disproportionality scores available • Powerful graphical visualizations to facilitate medical interpretation • Intuitive user interface including drill-down capabilities without requiring any programming • Built-in functionality to address confounding and assist in signal assessment such as stratification, logistic regression and extended logistic regression • Extensive built-in signal management, tracking and documentation functionality
  • 18. Empirica Signal Drug Profiles • Drug profiles support easy and consistent drug reviews – Presentations of statistical and other information about one or more drugs (generic, trade or ingredient) – Users don’t need to navigate Empirica Signal; instead they can review their drug using a pre-defined set of charts/graphs • Display can be in slide show mode or desktop review • Full power of data mining results, including drilldown, without having to navigate Empirica Signal • This graphical, easy to use review tool is directly accessible upon login without the need to issue any commands – Layouts define the charts (graphs or report outputs), options and positioning to be included – All graphs and tables can be printed directly according to the user-defined layouts – Access to publicly available labeling information (FDA, NIH)
  • 19. Comprehensive Drug Profile Layout
  • 20. Empirica Signal • Proactively identify and expeditiously manage safety risks • Detect and quantify safety signals by using advanced data mining techniques that can be applied to any spontaneous reporting database • Use the most advanced set of tools to develop a thorough and well understood safety profile
  • 21. Sector Map (Heatmap, Treemap) • Overall Safety Profile of a Drug • Large tiles are the MedDRA System Organ Class (SOC) • Smaller tiles are the Preferred Terms (PT) • Colors represent the strength of the signal – bright red indicates the AE’s occurring significantly higher than expected with the drug – bright green indicates significantly less occurrences than expected, or potential indications. • The chart below, is the ranking in descending order by the user selected disproportionality score – the algorythm in this example is the (EBGM – Empirical Bayes Geometric Mean) – also available are EB05,Chi or PRR. • The focus is on a particular drug, with the background being the 5M+ cases in the FDA AERS database
  • 22. Visual Presentation of Safety Signals In this map, you can clearly see the large number of AE’s in the Cardiac SOC as represented by the bright red color. This led to the 1998 market withdrawal of Seldane
  • 23. Sector Map • The size of PT tiles is based on the public health impact of PTs, which is computed as: (number of times the PT occurs in serious-or-fatal cases) * (proportion of cases with that PT that are serious-or-fatal). • PTs with values over a minimum threshold are included in the sector map. • A higher Public Health Impact score corresponds to a larger tile in the sector map. • Since size is based on the number of cases of the PT alone (not the number of cases that have the PT and a particular drug), the tile size is stable over sector maps for different drugs.
  • 24. Side by Side Comparison
  • 25. Zoom In on A Sector Map
  • 26. Drug Profile: Desktop View
  • 27. Drug Profile: Slide Show View
  • 28. Drug Profiles: Slide Show View
  • 29. Drug Profile: Slide Show View
  • 30. Drug Profile: Slide Show View
  • 31. Access to Safety Report Data All of the blue underlined items are hyperlinks to the underlying data The sub-menus are the same on the charts and graphs
  • 32. Access to Safety Report Data
  • 33. Download Safety Report Data
  • 34. Analytical Graphics for Safety Review
  • 35. Data Mining, Statistics and Data Visualization
  • 36. Query & Reporting Creating a Case Series with the Query Wizard Use selection options and/or saved lists After selecting variable values, edit the query logic to combine query elements
  • 37. Flexible Reporting and Tabulation
  • 38. Empirica Signal Management “Dashboard” • Supports “strategic pharmacovigilance,” helping drug safety groups prioritize work and focus resources on areas of greatest impact • Combines signal scores with metrics based on case counts and changes over time • Easy-to-use table and graphs interface with workflow and annotation tools
  • 39. Integrated single-table overview  Drug & event details  Signal statistics for multiple time points  Number of new cases in current period  Trending  50% increase in EBGM from last period  Current EB05 > EB95 measured previously  Indication of labeled event (“expectedness”)  Indication of FDA “sentinel event”  Percent of database cases with event that were fatal  Percent of database cases with event that were serious  Quadrant – A, B, C, D quadrant (Patrick Waller - MHRA)  Additional columns are easy to add
  • 40. Annotation • Annotations at the drug, drug-event combination and case levels • Permits both free-text and controlled vocabulary comments • Convenient way to build and share knowledge base
  • 41. Empirica Signal Capabilities • Data mining and statistics – Disproportionality statistics including empirical Bayes estimates of relative reporting rate (EBGM) and non-Bayesian statistics such as PRR and ROR – Logistic regression using adverse event as a response variable – Graphical and tabular presentation of results • Access to safety report data – Convenient drilldown to case data from graphs and tables – Download of signal scores and case data – Available for both public and private safety databases
  • 42. Empirica Signal Capabilities • Query & Reporting – Aggregate as well as line listing reports – Library of standard reports – Drilldown, download and publishing capabilities • Fully web-based application environment – All processing on the server – can be provided as hosted service – Simple web browser client – operates through firewalls
  • 43. Empirica Signal Management Capabilities (Optional) • Extends Empirica Signal functionality for routine ongoing review by safety evaluators • Combines disproportionality measures with other relevant information such as listedness, fatal outcomes, etc. to enhance the review process • Purpose: Timely detection of signals for marketed products while minimizing the overall monitoring effort • Capabilities include: – Easy detection of changes in signal scores – Structured on-line assessments – Prioritize, track and document signals – ‘Drill down’ to AE reports in underlying data
  • 44. Empirica Signal Topics Capabilities (Optional) • Provides workflow, auditing and tracking capabilities for potential signals • Supports topic attachments from multiple sources, either from inside the Empirica Signal application or from files or web site references • Provides topics workflow assignment and deadline capabilities • Topics can encompass multiple individual Topic Actions, each with their own description, attachments, assignees and deadlines • Topic visibility can be controlled in great detail • At any point in time a Topic can be represented in the form of a PDF or ZIP file for communication and archiving purposes
  • 45. Key Features • Customizable, easy-to-use Drug Profile Portal • Enhanced, configurable workflow • Support for a full range of statistical techniques • Enhanced, higher performance logistic regression • Bayesian Multi-item Gamma Poisson Shrinker interaction calculations for runs with three or more dimensions • Direct access to safety data through drilldown and case series features • Streamlined interface • Interactive reporting with parameter driven case selection • Options to download data and case details in native Microsoft Excel (.xls), .pdf, or rich text (.rtf) format • Robust, comprehensive online help
  • 46. Key Benefits • Identify and monitor safety trends using the same tools used by regulators • Prioritize and execute critical pharmacovigilance activities • Employ a variety of classical and Bayesian data mining techniques • Analyze both drug/event combinations and drug/drug interactions • Filter out false positives to focus on the most important risks • Detect problems that may have been overlooked during clinical development • Gather intelligence on competing drugs’ safety profiles • Maintain a comprehensive history of safety related topics • Track signals and manage actions in a regulatory complaint environment
  • 47. • Hosting • Training • Integration: Argus Argus Mart Empirica Signal Topics Perficient Offerings
  • 48. Empirica Signal Data Provisioning Services Subscriptions to periodic updates of prepared data for Empirica Signal • FDA FOI AERS data (1968 – present) Substantial data cleaning is conducted to prepare a proprietary version of FOI AERS safety data; these same cleaning techniques are also used to prepare the FDA-internal version of the AERS data for the FDA’s use with Empirica Signal: • Integrate incremental information at database and case level • Harmonize MedDRA coding of adverse events throughout the cumulative database across dictionary updates • Map verbatim drug descriptions to a controlled hierarchical vocabulary of trade and generic names (WHO-DD and WHO-ATC) • Identify and flag duplicate cases • FDA VAERS data (1990 – present) • Offered as a monthly or quarterly subscription • WHO-UMC Vigibase database of reports (1968 – present) • In-house corporate adverse event databases • Occasional spiking of in-house data into public sources
  • 49. Questions?
  • 50. References 1. Tetrazepam: ferral_prac_000015.jsp&mid=WC0b01ac05805c516f.htm 2. Xigris: 3. PropoxypheneDarvocet/Darvon): 4. Gemtuzumab ozogamicin (Mylotargz): 5. Rosiglitazone (Avandia): 6. Sibutramine (Reductil/Meridia): 7. Sitaxentan: 8. Efalizumab (Raptiva): 9. CIOMS VIII:
  • 51. Thank You! For more information, please contact: (Sales) +44 (0) 1865 910200 (U.K. Sales) +1 877 654 0033 (U.S. Sales)