Quantitative methods of signal detection on spontaneous reporting system databases - Seminaire Paris V

1. An agency of the European Union
Signal detection: Utilisation des
bases de l’OMS et de la FDA dans
le contexte de la detection des
signaux
Ne soyez pas dupes … je vais vous donner MON point de
vue
Presented by: François MAIGNEN
Principal scientific administrator (PhvRM)

2. Introduction & disclaimers
- Background (main objective of seminar)
- Conflicts of interests & disclaimer
- Apologies for the lack of French
- Learning objectives:
- What are the principles of disproportionality analysis +++
- Knowing your own data (size, characteristics incl. type of products, patients,
age of the database, quality and quantity of information, terminologies, etc
…)
- Knowing the other databases (characteristics, strengths, limitations)
2

3. Disproportionality analysis: it is a question of
data AND background!!!
3
3
Drug 1 All other
medicinal
products
Total
Event 1 a c
All other
reaction
terms
b d
Total
N = a +
b + c + d
c + d
a + c
a + b

4. Disproportionality analysis
• Refers to a particular method
• However, it is crucial to keep in mind that for a given drug-
event pair the result of the DA will be different from a
database to an other database
• The result of DA is valid AT A GIVEN POINT IN TIME
• Benefits of quantitative methods (threefold):
• Operational: easier to screen large databases
• Safety net against human error (to a certain extent): systematic way of
reviewing the information received in your database
• Time benefit in terms of discovery of the signals (possibly)
4

5. What are the elements to consider when doing
signal detection (in part. Quantitative meths)?
It is fundamental to know the database on which you are actually
doing the signal detection:
- Size: the benefit of using quantitative methods in small
databases is unclear
- Age: New or old database (type of products, terminology used,
legacy data, data fields and data model quantity of information
stored in the database, duplicates)
- Source of the data: spontaneous vs RCTs (or other sources),
type of reports (serious / non-serious, HCP vs patients)
- General characteristics: type of products (vaccines,
biologicals), patients, etc …5

6. Why using other databases?
• Other databases = other data sources (increase the likelihood
of finding new signals)
• Small databases: provide an opportunity to use an external
(spontaneous reporting) database as a background to
implement a DA approach
• Use of longitudinal (observational) databases (THIN, GPRD)
• Limitations: in particular cost, availability of information (FOI),
data management associated to it.
6

7. 7

8. 8

9. FDA AERS
• One of the oldest SRS database in the world: 1969
• Very large database: approx. 5 million reports
• HCP/consumers/Pharmaceutical Companies
• Post-authorisation
• 60% serious reports (i.e. 40% non serious)
• NO VACCINES
9

10. 10

11. 11

12. VAERS
- Established in 1990
- Medium size database: approx. 200,000 reports (30,000
reports / year)
- Maintained by FDA/CDC
- ALMOST UNIQUE: EXCLUSIVELY VACCINES
- Serious reports: 13%
- Hcp/Patients – spontaneous/RCTs
12

13. 13

14. WHO Vigibase
- The oldest database: established in 1968
- The largest: 7 million reports
- Mostly spontaneous reports incl. a small number of reports
involving vaccines
- Serious reports: < 10%
14

15. 15

16. EudraVigilance
- The newest of these databases, limited FOI
- Formally established in 1995, started in December 2001.
Implementation of mandatory electronic reporting in
November 2005.
- Large SRS: 2,200,000 reports
- Almost exclusively serious (EU), serious unexpected (non-EU)
- Contains both post-authorisation (spontaneous, observational,
registries, compassionate use) and clinical trials (RCTs)
modules
- New products incl. anticancer, antiretrovirals, biologicals. Over
representation of CAPs
16

17. Other (non spontaneous) databases …
• Longitudinal databases (GPRD, THIN, EU-ADR, etc …)
• Very useful for signal confirmation / signal strengthening:
Recent example (EMA): biphosphonates and risk of
cardiovascular valve disease
• Can be expensive, sometimes country specific
• Might “only” capture data from ambulatory patients (no
hospitalisation data)
• A lot of data management, not easy to analyse
• Lack of standardisation, pb of compatibility with SRS
databases (medical terminology, medicinal products)
17

18. 18
18
Use of longitudinal databases for signal
detection
Use of longitudinal databases (record linkage and electronic
health records – OMOP / Noren / Callreus) ~ incidence rate
ratio
• Same patients different time windows (A. Bate)
• Hospital records of different patients (T. Callreus)

19. 19
19
Merci Andrew

20. 20
20

21. 21
21
Interpretation / Limitations
Patients prescribed a PPI are associated with
acute pancreatitis in the month after the
Prescription (ICdiff positive)
But graph shows that these patients are general
likely to have acute pancreatitis around the time
of PPI prescribing (In agreement with confounding by
indication)

22. 22
22
Record linkage

23. 23
23

24. 24
24
Interpretation / limitations
Bias associated with the hospitalisation (confirmed by later
events which occurred remotely after the administration of the
vaccine)
Spurious / unexplained associations
Rely heavily on temporal association (Post hoc ergo propter
hoc)

25. 25
25
Longitudinal health records
Powerful to detect some associations
Even if reporting artefacts do not influence this method, not
devoid of other biases (selection, protopathic, misclassification,
etc …)
Much more complicated to implement (very large datasets,
confidentiality aspects, linkage of records and interoperability of
databases, etc …)

26. 26
26
Future directions (?)
•Performances in a prospective setting, added value of clinical
judgement needs to be quantified
•Other approaches to signal detection (use all the information
included in the reports, PK/PD properties of the substances)
•Surveillance networks (incl. spontaneous reporting) (Sentinel
initiative, ENCEPP, EU-ADR, etc …)

27. Summary
• Different databases: completely different characteristics
• Age (1968 – 2005)
• Type of products (vaccines, mixture of older and newer
products, overrepresentation of new products)
• Seriousness of the reports (<10% - 90%)
• Different types of databases: different types of signals,
different types of results for the DA
• CRITICAL TO KEEP THESE CHARACTERISITCS IN MIND WHEN
PERFORMING THE SIGNAL DETECTION ACTIVITIES +++
27

28. Merci
francois.maignen@ema.europa.eu
http://uk.linkedin.com/in/francoismaignen
@EMA_News
@FrancoisMaignen
@FrancoisMaignen/medicines
28

29. Conclusions
• Using external sources of information can be a (very) useful
adjunct to the signal detection activities
• The type of database (spontaneous reporting / longitudinal
database) important to define
• Know the characteristics of the database
• Operational issues: cost, compatibility between databases
29

30. Quizz 1
Question: Je suis en train d’etudier un produit specifique dans la
base de donnees de l’organisation pour laquelle je travaille
(approx. 10 nouvelles observations / semaine). Je n’ai pas
recu de nouveaux rapports d’effets indesirables pour ce
produit depuis plus d’un an. Le PRR de ce produit pour l’effet
que j’etudie calcule sur cette base de donnee est-il ?
- Reste inchange depuis la derniere annee
- A diminue
- A augmente
- A change
30

31. Quizz 2
Question: La limite inferieure de l’intervalle de confiance (95%)
du PRR pour un produit et une reaction (10 observations
d’effets indesirables) que j’etudie est egale a 0.85. Que puis-je
en conclure ?
- Je n’ai pas de signal associe a ce produit
- Je n’observe pas de signal de disproportionalite
- Je ne dois rien faire de plus avec cette combinaison produit-
reaction
31

32. Quizz 3
Question: Un collegue qui travaille pour une societe specialisee
(mais pas uniquement) dans des produits oncologiques se
demande pourquoi il n’observe jamais de SDR associe a des
agranulocytoses dans sa base de donnees. Que puis-je lui
conseiller ?
32

33. Quizz 4
Question: Je travaille pour une societe importante specialisee
dans les produits de biotechnologies. Je n’ai qu’un total de 30
observations spontanees pour un de mes produits dans ma
base de donnees (il s’agit d’une enzyme recombinante donnee
dans une maladie orpheline tres rare).
- J’aurais des problemes pour utiliser le PRR pour m’aider a faire
la detection des signaux pour ce produit
- Vu le nombre de notifications spontanees, ol est peu probable
que je trouve des signaux importants en terme de sante
publique
- Les type de produits que j’ai dans le reste de ma base de
donnees n’aura aucune influence sur la detection des reactions
d’immunogenicite avec le PRR.33

34. Quizz 5
Question: J’ai un probleme pour detecter et distinguer les
interactions medicamenteuses avec un produit dont je suis
responsible en utilisant le PRR. En pratique, quelles sont les
methodes qui sont a ma disposition pour essayer d’ameliorer
cette detection?
34