This document provides an introduction to signal detection in pharmacovigilance. It defines key terms like signal and signal detection. It describes qualitative and quantitative signal detection methods including individual case review, data mining algorithms to calculate disproportionality ratios from large safety databases, and visualization tools. Challenges with signal detection include determining what constitutes a "large enough" safety database and assumptions made when using spontaneous reporting data. The document outlines the signal management process from detection to prioritization and evaluation.
Signal detection is a process used in pharmacovigilance to identify potential safety issues or new safety information associated with a medicinal product. The goal of signal detection is to detect signals, or potential safety concerns, as early as possible in order to allow for timely risk management and safety interventions.
Signal detection typically involves analyzing large amounts of safety data, including adverse event reports, clinical trial data, post-marketing surveillance data, and other sources of safety information. The data is analyzed using statistical methods and algorithms to identify any patterns or trends that may suggest a potential safety concern.
Once a potential safety concern is identified, further investigation is typically required to confirm the signal and assess the magnitude of the risk. This may involve conducting additional studies, analyzing the available data in more detail, or consulting with regulatory agencies and other stakeholders.
Signal detection is an ongoing process that continues throughout the life cycle of a medicinal product. The process is critical for ensuring the ongoing safety and effectiveness of medicinal products, and is an important component of pharmacovigilance activities.
Introduction to Aggregate Reporting in Drug Safety & Pharmacovigilance in Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
Signal detection and management activities are at the core of ensuring drug safety. A complex process of signal detection; through their validation and confirmation; analysis and prioritisation; and signal assessment to recommending action.
Find out more at out training: http://bit.ly/1W31NCF
Signal detection is a process used in pharmacovigilance to identify potential safety issues or new safety information associated with a medicinal product. The goal of signal detection is to detect signals, or potential safety concerns, as early as possible in order to allow for timely risk management and safety interventions.
Signal detection typically involves analyzing large amounts of safety data, including adverse event reports, clinical trial data, post-marketing surveillance data, and other sources of safety information. The data is analyzed using statistical methods and algorithms to identify any patterns or trends that may suggest a potential safety concern.
Once a potential safety concern is identified, further investigation is typically required to confirm the signal and assess the magnitude of the risk. This may involve conducting additional studies, analyzing the available data in more detail, or consulting with regulatory agencies and other stakeholders.
Signal detection is an ongoing process that continues throughout the life cycle of a medicinal product. The process is critical for ensuring the ongoing safety and effectiveness of medicinal products, and is an important component of pharmacovigilance activities.
Introduction to Aggregate Reporting in Drug Safety & Pharmacovigilance in Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
Signal detection and management activities are at the core of ensuring drug safety. A complex process of signal detection; through their validation and confirmation; analysis and prioritisation; and signal assessment to recommending action.
Find out more at out training: http://bit.ly/1W31NCF
Identifying Safety Signals by Data Mining the FDA Adverse Event Reporting Sys...Perficient, Inc.
Ever since the European Union (EU) introduced new legislation that requires life sciences companies to proactively detect, prioritize, and evaluate safety signals, there has been an increased interest, not only from sponsors and CROs in the EU, but globally, in pharmacovigilance systems that can assist with the signal management process.
Perficient's Chris Wocosky, an expert in signal detection and management, shows how your organization can use Empirica Signal, Oracle's state-of-the-art signal detection system to data mine the existing FDA Adverse Event Reporting System (FAERS) to determine safety signals. This presentation and demonstration willhelp you bettter understand how this solution can be used in daily pharmacovigilance activities.
Pharmacovigilance Process Work Flow - Katalyst HLSKatalyst HLS
Introduction to Drug Safety & Pharmacovigilance Process Work Flow for Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
Literature Surveillance in Pharmacovigilance; Current Trends, Methods and Challenges
Please join Elizabeth E. Garrard, PharmD, founder and CEO of Garrard Safety Solutions, as she reviews key issues in literature surveillance for Pharmacovigilance.
Objectives:
• Understand the regulatory obligations, best sources and procedures for conducting literature surveillance.
• Appreciate some examples of when a safety signal was detected in the literature and its impact on the lifecycle of a drug.
• Understand when to start and where to look for emerging safety information.
• Setting up your search strategy, how to ensure your search strings are well balanced, recognizing the challenges between precision and sensitivity.
• What is the impact of the new literature monitoring by EMA of a number of substances in selected medical literature to identify suspected adverse reactions with medicines authorized in the European Union. Early insights into successes and issues.
• Discuss current methods that can increase the likelihood of early detection of a safety issue and minimize the issues surrounding.
• Realize the challenges we face including wide differences in quality, accuracy, and completeness in the scientific literature and how best to navigate these differences and maintain proper vigilance.
Introduction to ICSR Narrative Writing in Drug Safety & Pharmacovigilance in Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
History and Progress of Pharmacovigilance, Significance of Safety Monitoring, Pharmacovigilance in India And International Aspects, WHO International Drug Monitoring Programme, WHO and Regulatory Terminologies of ADR, Evaluation of Medication Safety, Establishing Pharmacovigilance Centres in Hospitals, Industry and National Programmes Related to Pharmacovigilance, Roles and Responsibilities in Pharmacovigilance, International Non-Proprietary Names for Drugs, International Classification of Diseases, Passive and Active Surveillance, Comparative Observational Studies, Targeted Clinical Investigations and Vaccine Safety Surveillance, Aris G Pharmacovigilance, VigiFlow, Statistical Methods for Evaluating Medication Safety Data
The aim of Safety reports is describe the safety during the lifecycle of the medicinal product. These reports are necessary during development as well as during the authorization process or renewal. In addition, several of these reports may be required by Health Authorities in case of safety concerns.
This presentation contains a full overview about periodic safety update reports and all the information related with it.
Identifying Safety Signals by Data Mining the FDA Adverse Event Reporting Sys...Perficient, Inc.
Ever since the European Union (EU) introduced new legislation that requires life sciences companies to proactively detect, prioritize, and evaluate safety signals, there has been an increased interest, not only from sponsors and CROs in the EU, but globally, in pharmacovigilance systems that can assist with the signal management process.
Perficient's Chris Wocosky, an expert in signal detection and management, shows how your organization can use Empirica Signal, Oracle's state-of-the-art signal detection system to data mine the existing FDA Adverse Event Reporting System (FAERS) to determine safety signals. This presentation and demonstration willhelp you bettter understand how this solution can be used in daily pharmacovigilance activities.
Pharmacovigilance Process Work Flow - Katalyst HLSKatalyst HLS
Introduction to Drug Safety & Pharmacovigilance Process Work Flow for Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
Literature Surveillance in Pharmacovigilance; Current Trends, Methods and Challenges
Please join Elizabeth E. Garrard, PharmD, founder and CEO of Garrard Safety Solutions, as she reviews key issues in literature surveillance for Pharmacovigilance.
Objectives:
• Understand the regulatory obligations, best sources and procedures for conducting literature surveillance.
• Appreciate some examples of when a safety signal was detected in the literature and its impact on the lifecycle of a drug.
• Understand when to start and where to look for emerging safety information.
• Setting up your search strategy, how to ensure your search strings are well balanced, recognizing the challenges between precision and sensitivity.
• What is the impact of the new literature monitoring by EMA of a number of substances in selected medical literature to identify suspected adverse reactions with medicines authorized in the European Union. Early insights into successes and issues.
• Discuss current methods that can increase the likelihood of early detection of a safety issue and minimize the issues surrounding.
• Realize the challenges we face including wide differences in quality, accuracy, and completeness in the scientific literature and how best to navigate these differences and maintain proper vigilance.
Introduction to ICSR Narrative Writing in Drug Safety & Pharmacovigilance in Pharmaceuticals, Bio-Pharmaceuticals, Medical Devices, Cosmeceuticals and Foods.
Contact:
"Katalyst Healthcares & Life Sciences"
South Plainfield, NJ, USA
info@KatalystHLS.com
History and Progress of Pharmacovigilance, Significance of Safety Monitoring, Pharmacovigilance in India And International Aspects, WHO International Drug Monitoring Programme, WHO and Regulatory Terminologies of ADR, Evaluation of Medication Safety, Establishing Pharmacovigilance Centres in Hospitals, Industry and National Programmes Related to Pharmacovigilance, Roles and Responsibilities in Pharmacovigilance, International Non-Proprietary Names for Drugs, International Classification of Diseases, Passive and Active Surveillance, Comparative Observational Studies, Targeted Clinical Investigations and Vaccine Safety Surveillance, Aris G Pharmacovigilance, VigiFlow, Statistical Methods for Evaluating Medication Safety Data
The aim of Safety reports is describe the safety during the lifecycle of the medicinal product. These reports are necessary during development as well as during the authorization process or renewal. In addition, several of these reports may be required by Health Authorities in case of safety concerns.
This presentation contains a full overview about periodic safety update reports and all the information related with it.
This training course on Drug Safety and Pharmacovigilance designed to give pharmaceutical and biologic companies operating in the U.S. and EU an understanding of product safety and regulatory compliance.
Revised by Christian Reina
Version: 1.1
Date: September 18, 2009
Change log:
-Risk Based Audit approach
-Things to know
-Penetration Testing Stages
-OSI Model protocols
-Firewall generations
-Wireless
-Common Criteria ISO 15408
-Problem Management
-System Development Life Cycle
-Software Life Cycle
-Five rules of evidence
-Incident Response framework
-Evidence Lifecycle
-Fair Information Practices
How AstraZeneca is Applying AI, Imaging & Data Analytics (AI-Driven Drug Deve...Nick Brown
Keynote AI Presentation given at AI-Driven Drug Development Summit Europe on 26th April 2023 in London. Overview around how AstraZeneca has been developing AI in the past 5+ years. Predominantly focused on R&D and how we are developing digital solutions & AI for right safety and right dose. AI examples include machine learning for safety assessment, augmenting digital pathology for image quantification & segmentation, understanding more about our drugs through advanced imaging modalities and first steps in applying AI for right dose - immunogenicity, adverse events and tolerability.
InvestigAction: a cognitive and organisational tool for learning from acciden...ALIAS Network
If you are interested in the topic please register to the ALIAS network:
http://network.aliasnetwork.eu/
to download other materials and get information about the ALIAS project (www.aliasnetwork.eu).
TargetInsights: A New Method to Rapidly Access 'Specificity” of Selected Prot...Ann-Marie Roche
In drug discovery, a lot of energy is spent on researching disease specific proteins which may be suitable targets for drug intervention. TargetInsights helps you to dig out all the information which is hidden beneath the title and the abstract.
Furthermore, Target Insights is a tool that helps you explore biological relationships in literature between concepts of interest. This webinar showed us how this is done.
Health Informatics – Application of Clinical Risk Management to the Manufacture and Deployment of Health Software. Thick M. eHealth week 2010 (Barcelona: CCIB Convention Centre; 2010)
Getting answers without asking questions at BHBIA WorkshopInSites on Stage
Getting answers without asking questions: Using social media netnography with patients (by Robert Dossin), presented at the BHBIA Workshop in London (UK) on Thursday January 17, 2013.
Threat Hunting 101: Intro to Threat Detection and Incident ResponseInfocyte
Join Infocyte's Vice President of Customer and Partner Success, Chris Mills, for Threat Hunting 101: An intro to using Infocyte HUNT to detect, investigate, and respond to advanced persistent threats, file-less malware, and other sophisticated attacks.
Beyond these slides, please reference the video for additional insight and instruction on how to use our Threat Hunting and Incident Response platform.
Similar to Introduction to Pharmacovigilance Signal Detection (20)
2013 OHSUG - Facilitating Pharmacovigilance Globalization with Process Reengi...
Introduction to Pharmacovigilance Signal Detection
1. Introduction to
Signal Detection
by Rodney L. Lemery MPH, PhD
Copyright BioPharm Systems, Inc. 2011. All rights reserved.
2. Topics…
• Definitions
– Pharmacoepidemiology
– Pharmacovigilance
– Signal
– Signal Detection
• Signal Detection
– Qualitative
• Striking cases
• Periodic reviews
– Quantitative
• Analysis of disproportionality
Introduction to Signal Detection 2
3. Topics…
• Signal Detection (cont.)
– Disproportionality analysis tools
• Signal Prioritization
– WHO Triage
– MHRA Impact Analysis
– Categorization of priority
• Confirmed Signal
• Unconfirmed Signal
• Refuted Signal
• Signal Evaluation
– Causality
– Frequency
– Clinical implications
– Preventability
Introduction to Signal Detection 3
4. Acronyms and General
Definitions
• Prevalence – The total number of cases of a disease in
a given population at a specific time
• Incidence – The number of newly diagnosed cases
during a specific time period
• EMA – European Medicines Agency
• ADR – Adverse Drug Reaction
• APR – Adverse Product Reaction
• ICH – International Conference on Harmonization
• CIOMS – Council for International Organizations of
Medical Sciences
Introduction to Signal Detection 4
5. Definitions…
• Two pervasive definitions (Abenhaim, Moore,
& Begaud, 1999)
1. The collection and scientific evaluation of
adverse drug reactions (ADR), under normal
conditions of use for regulatory purpose.
−Restricts the concept to regulatory
Pharmacovigilance compliance only and only medicinal
products.
2. Watchfulness in guarding against danger from
products or providing for safety of the product
−Expansive beyond just regulations and
frames the construct for use in academia
and the sciences
Introduction to Signal Detection 5
6. Definitions…
•The application of epidemiologic
techniques used to study the effects of
Pharmacoepidemiology drugs in populations
−First mentioned in the early 1980’s
(Abenhaim, Moore, & Begaud, 1999)
Introduction to Signal Detection 6
7. Exercise 1
1. Please segregate into your
Define the small group as assigned by the
index card in your chair
following terms: 2. Using collaborative discussion,
please reach consensus for the
definition of the following terms:
Signal “Signal”
“Signal Detection”
3. Nominate a single
spokesperson from the group to
Signal Detection share the definition with the
larger audience.
Introduction to Signal Detection 7
8. Definitions…
•Much debate on the definition (we will use the
following):
•Information that arises from one or multiple sources,
which suggests a new potentially causal association, or
Signal a new aspect of a known association, between an
intervention and an event or set of related events, either
adverse or beneficial, that is judged to be of sufficient
likelihood to justify verificatory actions. (CIOMS, 2010
p.14)
Introduction to Signal Detection 8
9. Definitions
•Much debate on the definition (we will use the
Signal following):
•The act of looking for and/or identifying signals
Detection using event data from any source . (CIOMS, 2010
p.116)
Introduction to Signal Detection 9
10. Simplified Safety Signal Lifecycle
Signal Signal
Detection Prioritization
Signal
Evaluation
CIOMS (2010, p. 9)
Introduction to Signal Detection 10
11. Detailed Signal Management Process
Signal
Detection
Signal
Prioritization
Signal
Evaluation
11
Introduction to Signal Detection
13. Individual Case Safety Reports (ICSR)
• The accumulation of ICSR can occur from multiple
places according to the ICH E2D Guideline (ICH, 2003)
Sources of ICSR Description of Sources
Unsolicited sources Spontaneous reporting
Solicited sources Any organized collection of data
(outcomes research, clinical trials,
registries, surveys, billing databases
etc.)
Contractual agreements Inter-company exchange of safety data
Regulatory authority Any ICSR originating from the
regulatory authority submitted to a
company
Introduction to Signal Detection 13
14. Signal Detection…
Traditional Pharmacovigilance
Methods
· Individual case review
· Aggregate analysis
· Periodic reports
• Traditionally, signals are detected through the
assessment of individual case safety reports
(ICSR) in an individual or cumulative manner
Introduction to Signal Detection 14
15. Signal Detection…
• Qualitative review of ICSR
– Critical method of detection for events where the
background incidence is uncommon or rare and should
not be replaced by quantitative methods (Egberts,
Meyboom, & van Puijenbroek, 2002; Hopstadius,
Norény, Bate, & Edwards, 2008)
– May be implemented as a list of designated medical
events (DME) within a company
• Often manifest in systems as custom Standardized MedDRA
Queries (SMQ)
– FDA has a list of “Interesting PTs”
– EMA follows serious events identified by CIOMS V
• Results in the identification of “index” or “striking”
cases that can be monitored
Introduction to Signal Detection 15
16. Signal Detection…
• Qualitative review of ICSR (cont.)
– Periodic review of case series
• PSUR or equivalent
– Review of the aggregate tables displayed in
these periodic reports can provide indications of
potential “striking” events
• Specific sections such as the “Overall Safety
Evaluation” provide clinical context around the
identification of such signals and become an
important part of signal detection
(Waller, 2009 p. 67)
Introduction to Signal Detection 16
17. Exercise 2
1. Please segregate into your
small group as assigned by the
index card in your chair
2. Using collaborative discussion
and the DSUR provided, please
DSUR AE determine the presence or
Summary Table absence of striking cases in the
table.
Review Why did you or didn’t you
identify cases of interest?
3. Nominate a single
spokesperson from the group to
share the results with the larger
audience.
Introduction to Signal Detection 17
18. Signal Detection…
• Quantitative assessment of ICSR
– Absolute counts
• Number of reports of an adverse event (AE) or
adverse product reaction (APR)
985
No APR All completed the 1 year
1000 duration of study resulting in
Participants 985 person-years
in an IND
study Steven’s Johnson
Syndrome (SJS)
15
Completed the 6 month mark
of study resulting in 7.5
person-years
Introduction to Signal Detection 18
19. Signal Detection…
• With a total of 992.5 person-years for the denominator of the
study incidence rate, we can calculate the incidence rate of SJS
in the IND population as 15/992.5 which is 0.0151 cases/992.5
person years
– A typical denominator is million person years so we must convert or
IND incidence rate to million person years
• In terms of million person years this becomes 15.1 cases/million person
years
• Using epidemiologic sources, we note that the estimated
background incidence rate of SJS is 1-6 cases/million person
years
• The incidence rate in the study is suspiciously higher than the
expected background rate therefore SJS is a signal
Introduction to Signal Detection 19
20. Signal Detection…
• Quantitative assessment of ICSR (cont.)
– Proportions
• Number of specific AE reports divided by total number of
reports for a given product
– Assume an IND study of 100 patients for a new
Anticonvulsant
– At the end of the study there are 450 ICSR for the product
– Within this 450 are 25 reports of “GI bleeding”
– The proportion would be 25/450 or ~6%
• NOTE: True “expected” proportions are not really possible to
determine from spontaneous reporting data
– However, highly prevalent product/event
combinations for given case series may
be enough to elevate the combination
to a signal
Introduction to Signal Detection 20
21. Exercise 3
Determine the
following with the 1. Please segregate into your
small group as assigned by the
data distributed: index card in your chair
2. Using collaborative discussion
and the information provided,
Is there an please calculate the study
prevalence for “Peripheral
indication that neuropathy” and compare it to
“Peripheral the provided epidemiologic data
neuropathy” may Has a signal been identified
in the data?
have a relationship 3. Nominate a single
to the spokesperson from the group to
consumption of share the results with the larger
audience.
“Alfenta”?
Introduction to Signal Detection 21
22. Disproportionality…
Data Mining Algorithms
· Disproportional reporting
ratios
• Once the ICSR database becomes large enough*, statistical techniques
(generally referred to as data mining) can be applied
– Usually on large datasets from regulatory agencies or public health
entities
– WHO: Vigibase
– FDA: AERS
– MCA: Sentinel (formally known as ADROIT)
– EMA: EudraVigilance
– Generally these techniques identify
disproportionate reporting ratios
*”Large Enough” is a function of product/event incidence in the population
Introduction to Signal Detection 22
23. “Large Enough”
• Implications on population size for sampling in signal
detection can be reduced to the following (CIOMS, 2010
p.31) :
Event Background Example Ease of Signal Detection
Incidence Incidence proving an Method
in Product of Event association
Takers (method)
Common Rare Phocomelia Easy ICSR or Periodic
due to Thalidomide (clinical observation) Review
Rare Rare Reye’s syndrome Less easy ICSR or Periodic
and Aspirin (clinical observation) Review
Common Common Cough Difficult Data Mining
and ACE inhibitors (large observational
trials/data)
Uncommon Common Breast carcinoma Very difficult Data Mining
to Rare and Hormone (large clinical trials)
Replacement
Therapies
Rare Common None known Virtually impossible Virtually impossible
Introduction to Signal Detection 23
24. Classical Versus Bayesian…
• The basis of disproportionality analysis is
either frequentist statistics or Bayesian
(Gajewski & Simon 2008)
– Classical (frequentist) statistics look at probabilities as long
term frequency with an assumption of repeatable
experiment or sampling methods and a “true” value for a
parameter
• TOTAL INFORMATION=Data from experimentation
– Bayesian statistics look at “true” probabilities as a function
incorporating prior beliefs or knowledge and is updated
• TOTAL INFORMATION=Historical Information + Data from
experimentation
Introduction to Signal Detection 24
26. Data Mining Algorithms…
• All measures calculated from a 2X2 Contingency Table
– Classical
• Proportional Rate Ratio (PRR)
• Reporting Odds Ratio (ROR)
• Relative Reporting Ratio (RRR)
– Bayesian
• Information Component
• MGPS/EBGM
Event of All other Events TOTAL
Interest
Product of Interest A B A+B
All other Products C D C+D
TOTAL A+C B+D A+B+C+D
Introduction to Signal Detection 26
27. Data Mining Algorithms…
• All measures of disproportionate reporting are
basically calculations of
OBSERVED/EXPECTED
• In PV, the EXPECTED data is also referred to
as the “background”
• What you include in the “background” is a point
of contention in the industry and no real rules
are present (Gogolak, 2003)
Introduction to Signal Detection 27
28. Data Mining Algorithms…
• Since the simple calculation is O/E, the
relationship between background and the
statistic of interest is inversely related:
– As the background increases the resulting
statistic decreases
• Large E results in small PRR
– As the background decreases the
resulting statistic increases
• Small E results in large PRR
Introduction to Signal Detection 28
29. Data Mining Algorithms
• In general, the frequentist or Bayesian methods will
perform similarly when there are five or more reports
of a particular product-event pair (CIOMS, 2010, pp.
59)
• Bayesian methods may provide a solution to “false
positive” indications in large datasets
• It is important to note that the literature does not
demonstrate consensus on cost/utility of various data
mining tools with respect to specificity and sensitivity
– (CIOMS, 2010, pp. 61)
• As such, company specific decisions
must be discussed as part of the
signal management process
Introduction to Signal Detection 29
30. Technical Solutions…
• Use of these databases requires that certain
assumptions be made
– Drugs used in the marketplace are used by a
representative sample of the greater population
• Any information derived from these databases should
be interpreted using the limitations of the data
contained therein (Edwards, 1999)
– Limited clinical quality of data
• USA allows reporting into the AERS system from anyone
(Health care provider {HCP} or not)
• EMEA only allows reporting by HCP thus typically more
complete clinical information
Introduction to Signal Detection 30
31. Technical Solutions…
• Three main products are available for large data
mining opportunities
– Qscan by Drug Logic
– Empirica Signal by Oracle
– agSignals by Aris Global
– dsExplorer by Cerner
• In the spirit of full-disclosure, BioPharm Systems is a
Gold Partner with Oracle and the next slides are taken
from the Empirica Signal product
– Please note that except for very specific functional differences,
these software systems are designed to accomplish similar
tasks. The degree to which one is better than the other is not
discussed in this presentation and the use of slides is not an
endorsement of one product over another
– As already stated, the decision to evaluate
signals and the method used is a company
specific decision
Introduction to Signal Detection 31
32. Technical Solutions…
• Empirica Signal can display disproportionality
results in a sector map fashion which allows for
visual assessment of signal strength.
Introduction to Signal Detection 32
33. Technical Solutions…
• In this example of
“Ziconotide”, we see
that sector 2 is
“Vertigo” with a
disproportionality
ratio of 2.052
– This is a good example though
of a well know substantiated
concern with the product as it
is known to disturb motor
function and balance
Introduction to Signal Detection 33
34. Technical Solutions…
• Sector 1 is “Tinnitus”
with a disproportionality
ratio of 4.194
Introduction to Signal Detection 34
35. Technical Solutions…
• The software solutions available provide a
wonderful way to quickly and comprehensively
analyze marketed data reported to regulatory
agencies.
• These data are subject to the issues surrounding
the collection of the information in them
– Underreporting of serious events
• Changes the number of expected events
• “Weber Effect”: The peak reporting for events in a drug on
market occurs within the first 2 years of approval (Hartnell, &
Wilson, 2004) during the initial 5 year marketing period
– Over reporting of events of
non-interest (expected non-serious)
Introduction to Signal Detection 35
36. Technical Solutions…
• False Causality attribution
– Signals ARE NOT CAUSAL INDICATIONS
– They are disproportionate reporting
indicators
• Mitigation of these limitations can occur if you
establish a signal prioritization method in your
company for dealing with the various signals
identified by manual or automated signal
detection
Introduction to Signal Detection 36
37. Signal Prioritization…
Triage of Outputs
· Interpret the signal in context of other
relevant sources, disease knowledge,
biologic plausibility, alternative
etiologies, etc.
Monitor
If signal is
indeterminate
NO
Impact Is signal
assessment and Need further NO
YES investigation refuted?
prioritization
Close out
YE
S
(if signal is refuted)
Introduction to Signal Detection 37
38. Signal Prioritization
• Prioritization of signals is still a very
controversial aspect of signal
management (Waller, 2010 p. 50)
• Two proposed methods are as follows:
– WHO – Triage (CIOMS, 2010 p. 88;
Lindquist, 2007)
– MHRA – Impact analysis (draft literature)
Introduction to Signal Detection 38
39. WHO - Triage
• Seriousness assessment
– Is the event serious or not?
• Unexpectedness
– Is the event expected or not?
• Disproportionality score
– Is the score high or not?
• Temporal displacement of score
– Has the disproportionality score increased over
time?
Introduction to Signal Detection 39
40. WHO - Triage
• Temporal occurrence
– Is the event occurring within the first few years of
launch
• Careful as Weber effect could contribute to
confounding here
• Multiple signaling countries
– Are more than one country seeing this
issue?
• Positive Rechallange
– Is there evidence of positive rechallange?
• Specialty list of terms of interest
– Is this a term of interest as identified by the
company
Introduction to Signal Detection 40
41. MHRA – Impact analysis…
• A calculated quantitative score based
on “Evidence” and “Public Health”
– Evidence Score
• Degree of disproportionality (PRR, IC etc)
• Strength of evidence
• Biologic Plausibility
– Public Health Score
• # of reported cases per year
• Expected health consequences
• Reporting rate in relationship to the
level of drug exposure
Introduction to Signal Detection 41
42. MHRA – Impact analysis
• Results in the following quantitative
categories:
– High
– Need to gather more information
– Low
– No action
• Additionally the MHRA is looking at a method to
incorporate the following in a prioritization
scheme:
– High profile product (media
attention)
– Risk perception by general
population
– Political obligations
Introduction to Signal Detection 42
43. Signal Prioritization
• Regardless of the method used, every
signal should undergo this type
prioritization
– CIOMS (2010 p. 22) suggest that this effort
results in the following outcomes which I have
labeled in the following manner:
• Confirmed signal
– Results in the movement of the signal to the evaluation
process
• Unconfirmed signal
– Results in the monitoring of this event over time and regularly
re-prioritizing it based on new information
• Refuted signal
– Results in the closing of the signal
» Disease progressions
» Known issues etc.
Introduction to Signal Detection 43
44. Signal Prioritization
• Regardless of the method used, care
should be taken in that this type of
assessment and analysis
should be an iterative process
when new information is
amassed
Introduction to Signal Detection 44
45. Exercise 4
1. Please segregate into your
small group as assigned by the
index card in your chair
2. Using collaborative discussion
and the information provided,
Signal please prioritize the various
Prioritization signals using one of the above
methods (WHO or MHRA)
3. Nominate a single
spokesperson from the group to
share the findings with the larger
audience.
Introduction to Signal Detection 45
46. Signal Evaluation…
Signal Evaluation
· Individual case review
· Aggregate analysis
· Periodic reports
· Non-interventional studies
· Non-clinical studies
· Class analysis
· Other relevant information
Introduction to Signal Detection 46
47. Key Components to Signal
Evaluation…
• Once a signal is prioritized as
“Confirmed” or its equivalent, further
data must be gathered to further the
evaluation of the signal (Waller, 2010 p.
50, 51)
• Causality
– Does the provided evidence support a causal
relationship between the product and event?
• ICSR Causal Relationships
– Probable, possible, unlikely, unrelated, unassessable
Causality is more than this
Introduction to Signal Detection 47
48. Causality…
• Bradford-Hill (Shakir & Layton, 2002)
proposed the following categories should be
used to assess causal relationships found in
data
– Strength
• The stronger an association, the less likely it is
explained by other factors
– Consistency
• Multiple sources demonstrate similar
associations
– Temporality
• Is the event consistently treatment
emergent?
– Positive re or de challenge evidence
Introduction to Signal Detection 48
49. Causality…
• Bradford-Hill (cont.)
– Biologic Gradient
• Evidence of dose or duration related risk
– Dose-dependency or cumulative exposure over time
– Specificity
• Single product exposure results in event
– Presence of this leaves little doubt to causality
– A few adverse drug reactions in an of themselves are
syndromes
• Absence of this does not reflect a non-causal
relationship
– For example nicotine exposure and
lung cancers
Introduction to Signal Detection 49
50. Causality…
• Bradford-Hill (cont.)
– Plausibility
• Does the existing literature support this association?
– Coherence
• Is the association compatible with the existing literature and
knowledge?
– Experimental Evidence
• Is there data that demonstrates the event can be altered or
eliminated by some experimental regimen?
– Converging evidence between post marketed and clinical surveillance
– Analogy
• Are there existing alternative
explanations for the association
in the literature?
• Absence of these strengthens the causal likelihood
• Absence of any or all of these four criteria does not
indicate the absence of a causal relationship since
out data may be describing a newly
seen observation not part of the
literature used in these assessments
Introduction to Signal Detection 50
51. Causality…
• In general, the more of the criteria satisfied,
the stronger the causal relationship. The
assessment of how many and which criteria
are more important than others, is no simple
formula and judgment is required (Waller,
2010 p. 29)
• There are issues with the PV databases and
literature used in signal evaluation and one
must keep these in mind when applying the
Bradford-Hill criteria (Shakir &
Layton, 2002)
Introduction to Signal Detection 51
52. Key Components to Signal
Evaluation…
• In addition to assessments of causality, the
concept of Frequency is important in signal
evaluation (Waller, 2010 p. 51)
– The question of frequency can be categorized
into the following measures of prevalence
• Very Common
– More than 1:10
• Common
– 1:10 to 1:100
• Uncommon
– 1:100 to 1:1000
• Rare
– 1:1000 to 1:10,000
• Very Rare
– Less than 1:10,000
Introduction to Signal Detection 52
53. Key Components to Signal
Evaluation…
• In addition to assessments of causality and
frequency, the concept of Clinical
implications is important in signal evaluation
(Waller, 2010 p. 51)
– Clinical Implications can be rephrased to impact
on patient health
• Is the event life-threatening?
• Does its presence lead to a congenital anomaly?
• Does its presence lead to death?
• Does its presence lead to long term
disability or long term hospitalization?
Introduction to Signal Detection 53
54. Key Components to Signal
Evaluation…
• In addition to assessments of causality,
frequency and clinical implications, the
concept of Preventability is important in
signal evaluation (Waller, 2010 p. 51)
– Preventability
• If an intervention could be applied at this stage, would
the event of interest or its outcomes be prevented?
Introduction to Signal Detection 54
55. Key Components to Signal
Evaluation…
• To summarize, every confirmed signal is
evaluated in terms of the following:
– Causality
– Frequency
– Clinical implications
– Preventability
• Based on the suspected signal’s evaluation
outcome, you may choose to elevate it to the
status of a potential or identified
risk (CIOMS, 2010 p. 93, 94)
Introduction to Signal Detection 55
56. Key Components to Signal
Evaluation
• This signal evaluation process is intimately
tied to risk identification and may result into
the feeding of the signal into the risk
mitigation/management planning processes
at your company.
• The information collected in the signal
evaluation phase will feed the safety
specification section of the risk management
plan.
Introduction to Signal Detection 56
57. Exercise 5
1. Please segregate into your
small group as assigned by the
index card in your chair
2. Using the provided clinical
information, is there evidence
Signal Evaluation that “Ascites” should be
elevated to a risk?
3. Nominate a single
spokesperson from the group to
share the findings with the larger
audience.
Introduction to Signal Detection 57
58. Summary
1. Create a procedure to formalize signal
management in your organization
• Ensure your process includes signal detection,
prioritization and evaluation
2. Signal detection efforts should include both
qualitative and quantitative methods
Top Five 3. Signal prioritization should describe the objective
methods used to categorize the signals
Take- discovered through detection efforts
Aways 4. Signal evaluation should be used to document
and formally evaluate only those critical signals
(from prioritization methods)
5. Information collected from your formal evaluation
strategies can be used to seamlessly parse into
your risk management process
Introduction to Signal Detection 58
59. References
• Council for International Organizations of Medical Sciences (CIOMS). (2010). Practical Aspects of Signal
Detection in Pharmacovigilance. Report of CIOMS Working Group VIII, Geneva .
• Egberts, A.C.G., Meyboom, R.H.B., and van Puijenbroek, E.P. (2002). Use of Measures of Disproportionality in
Pharmacovigilance: Three Dutch Examples Drug Safety 25(6): 453-458
• Gajewski, B.J. and Simon, S.D. (2008). A One-Hour Training Seminar on Bayesian Statistics for Nursing
Graduate Students. The American Statistician, 62 (3)
• Hartnell, N.R. and Wilson, J.P.. (2004). Replication of the Weber Effect Using Postmarketing Adverse Event
Reports Voluntarily Submitted to the United States Food and Drug Administration. Pharmacotherapy. 24:743-
749
• Hopstadius, J., Norény, G.N., Bate, A., and Edwards, R.. (2008). Impact of Stratification on Adverse Drug
Reaction Surveillance. Drug Safety. 31(11)
• International Conference on Harmonisation (ICH). (2003). ICH Harmonised Tripartite Guideline. Post-approval
safety data management: Definitions and standards for expedited reporting E2D. Retrieved on July 26. 2011
from
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E2D/Step4/E2D_Guideline.pdf
• Levitan, B., Yee, C. L., Russo, L., Bayney, R., Thomas, A. P., and Klincewicz, S. L. (2008). A Model for
Decision Support in Signal Triage. Drug Safety, 31 (9)
• Lindquist, M. (2007). Use of triage strategies in the WHO signal-detection process. Drug Safety, 30:635-7
• Maggid. (2011). Retrieved from http://actuary-info.blogspot.com/2011/05/homo-actuarius-bayesianes.html on
August 31, 2011
• Shakir, A.W., and Layton, D.. (2002). Causal Association in Pharmacovigilance and Pharmacoepidemiology:
Thoughts on the Application of the Austin Bradford-Hill Criteria. Drug Safety. 25 (6): 467-471
• Strom, B. L., and Kimmel, S. E. (2006). Textbook of Pharmacoepidemiology. Wiley, West Sussex, UK
• Waller, P. (2010). An Introduction to Pharmacovigilance. Wiley-Blackwell. Oxford, UK
Introduction to Signal Detection 59
60. Contact Information
Rodney has over 15 years experience in clinical research including
laboratory experimentation, clinical data management, clinical trial
design, dictionary coding and pharmacovigilance.
Rodney has worked for BioPharm Systems for eleven years now
serving in a variety of roles all related to the technical and/or clinical
implementations of software systems used in the clinical trial
process.
Prior to coming to BioPharm Systems Rodney worked at
pharmaceutical and technology companies in the Dictionary Coding,
Statistical Programming and Data Management areas.
In addition to his current work at BioPharm Systems, Rodney holds
an Associate faculty position at Walden University teaching Public
Health Informatics and other health information systems courses.
Rodney holds a Bachelor of Science in Genetic Engineering, a
Masters of Public Health in International Epidemiology and a Ph.D. in
Epidemiology focusing on Social Epidemiology
Introduction to Signal Detection 60