Understand the role investigators and medical monitors play in determining the safety of new drugs and biologics in development.

1. Identifying Safety Signals in
Product Development
Pharmacovigilance

2. Jack Modell, MD, VP and Senior Medical Officer
Dr. Modell is a board-certified psychiatrist
with 35 years’ of experience in clinical
research and patient care, including 15
years of experience in clinical drug
development in the pharmaceutical
industry. He has led successful
development programs in the
neurosciences, is a key opinion leader in
the neurosciences, has served on numerous
advisory boards, and is nationally known for
leading the first successful development of
preventative pharmacotherapy for the
depressive episodes of seasonal affective
disorder.

3. Sponsor Monitoring of Clinical
Investigations
• Title 21 - CFR 312* Sponsor shall
– Monitor the progress of all clinical investigations being conducted
under its IND (21CFR 312.56)
– Review and evaluate the evidence relating to safety and effectiveness
of the drug as it is obtained from the investigator (21CFR 312.56)
– Report to the FDA in an ongoing manner certain information relevant
to safety, most prominently serious unexpected adverse events
associated with the study drug. (21CFR 312.32)
– Submit a summary of all safety information obtained during the
previous year in an annual report (21CFR 312.33)
*21: Food and Drugs; 312: IND
Examples of other CFR titles: 3: The President; 8: Aliens & Nationality; 14: Aeronautics &
Space; 20: Employee’s Benefits; 33: Navigation & Navigable Waters; 50: Wildlife & Fisheries

4. Safety Monitoring of Clinical
Trials
• Sponsor has two responsibilities during trials:
– Continuous monitoring of trials to detect safety
signals requiring prompt attention, reporting and
possible action
– Regular periodic analysis and reporting of safety
information (during development and marketing)

5. Safety Monitoring of Clinical
Trials
Investigator has one primary safety
responsibility during trials, which he/she often
overlooks: to objectively assess and report all
safety information that emerges during the
course of the clinical trial.

6. Investigator Responsibility
What investigators must always remember is that the purpose
of clinical research is not to show that a drug is safe; it is not
to protect the drug from unfairly being considered unsafe; it is
not to show efficacy (remember that the null hypothesis on
which our research is based is that drug = placebo, which
should be the viewpoint from which we assess safety and
efficacy); and it is certainly not to show fewer AEs rather than
more, or to minimize the categorization of AEs as possibly
drug related.

7. Investigator Responsibility
• We must patient safety first, which includes the safety of
patients who might get the drug in the future.
• For that reason, it is generally better to err on the side of
calling an untoward or undesirable event an AE than not to.
Almost every marketed product has published AE rates that
are a fraction of what is seen in clinical practice.
• Having data that greatly underestimate actual AE rates causes
a lot of patient anguish, and sometimes harm, especially
when patients and their physicians didn’t suspect a newly
developed sign or symptom was drug related because of the
low rates reported in clinical trials.

8. Investigator Responsibility
• FDA has noted that investigator assessment of IP-attribution is
often inaccurate, and would rarely determine an overall safety
assessment of a product. In many studies, up to half of all AEs
that occur on placebo are attributed by investigators to be at
least possibly IP-related, when most couldn’t have been.
• FDA does their own assessment of what’s likely drug related
based on unblinded data in light of the drug’s mechanism of
action, known properties of related drugs, etc.
• The potential implications of making the wrong decision, for
the safety of the study subject in question, the product’s
approval, and the safety of future patients who will take the
drug, are generally greater by “under-calling” potential AEs
than “over-calling” them.

9. Safety Processes
• Ongoing medical monitoring for risk
– Sponsor and/or CRO Medical Monitor
– Sponsor and/or CRO Safety Department
• Data safety monitoring committee/board (DSMC/B)
– Independent experts review safety data periodically
– Can recommend changes to trial, interruption or
discontinuation if “safety signal” is detected
– No FDA regulations require a DSMC, but often
recommended, especially for large trials and/or those
involving potential significant safety risks.

10. FDA Guidance for Industry Premarketing
Risk Assessment (March 2005)
• Clinical risk assessment in drug development
should be guided by
– Preclinical safety assessments
– Clinical pharmacology program
• Understanding of metabolic pathways
• Possible drug-drug interactions
• Effects of hepatic and/or renal impairment
But pre-clinical information is not fully predictive of effects in humans,
so need to look for “signals” in our studies.

11. What is a Signal?
• General meaning is something that:
1. conveys information
2. incites to action
• In pharmacovigilance:
– An unexpected frequency of adverse events,
specific type of adverse event(s), or pattern of
adverse events for the known characteristics of a
product in a given population

12. What is a Signal?
• Council for International Organizations of
Medical Sciences (CIOMS) definition:
– “Information that arises from one or multiple
sources (including observations or experiments),
which suggests a new, potentially causal
association, or a new aspect of a known
association between an intervention [e.g.,
administration of a medicine] and an event or set
of related events, either adverse or beneficial, that
is judged to be of sufficient likelihood to justify
verificatory action.”

13. Safety Information in Early Trials
• Important for identifying issues to be
evaluated further in later studies:
– “Events of special interest”; e.g., events suggestive
of adverse drug effect not suspected from pre-
clinical data
– Potential concerns for special populations (e.g.
elderly)
• Comprises part of overall safety information
presented to FDA in the NDA

14. Early Trials Safety Signals
• Critical to identify concerns for subjects in the
current trial before dosing additional subjects
at a given level or escalating to higher dose or
longer dosing
– If sponsor determines that its investigational drug
presents an unreasonable and significant risk to
subjects, it must discontinue the investigations
presenting the risk and notify the FDA and all IRBs
and investigators involved. (21 CFR 312.56)

15. Definitions & Guidance under
Amended 21 CFR 312.32
• Adverse event (AE): any untoward medical occurrence
associated with the use of a drug in humans, whether or not
considered drug related
• Serious Adverse Event (SAE): an AE, which in the view of
investigator or sponsor, results in:
– death, life-threatening adverse event, inpatient hospitalization or
prolongation of existing hospitalization, persistent or significant
incapacity or substantial disruption of the ability to conduct normal
life functions, congenital anomaly/birth defect; or may require
intervention to prevent one of these.

16. Adverse Events in Phase 1 FTIH
trials
• May not be any “expected” AEs for unique new compound if
no previous reports in humans
– Common symptoms (e.g. headaches) will occur normally in some
subjects at some point during a trial
• SAEs should not occur (except for events outside of the study
which are clearly unrelated to study product)
• Patterns are difficult to detect in small numbers of subjects,
so any occurrence of unusual or severe signs/symptoms
warrants attention
– Modified from presentation by Judy Racoosin MD, MPH; Senior policy
advisor in the FDA Office of the Center Director, CDER. April 2009,
Philadelphia.

17. Causes of adverse events
• Investigational drug
• Study procedures
• Concomitant medications
• Disease under study or another incident
disease
• Ascertainment bias
• Accidents, injuries
• Normal variation

18. Causality and Expectedness
• PI makes an assessment of whether an AE is causally related
to the investigational product. Often difficult to assess drug-
relatedness of an adverse event in an individual case report.
• For SAEs:
– medical monitor and/or sponsor assess whether this event is expected
based on what has already been demonstrated to be associated with
the product.
– Unexpected: nature or severity of event is not consistent with
information about previously observed reactions in the Investigator’s
Brochure (or the package insert/label for approved products).
“If I hadn’t believed it, I never would have seen it!”

19. The Bradford Hill criteria, otherwise known as Hill's criteria for causation, are a
group of minimal conditions necessary to provide adequate evidence of a causal
relationship between an incidence and a possible consequence, established by
the English epidemiologist Sir Austin Bradford Hill (1897–1991) in 1965.
Austin

20. Bradford Hill Criteria for
Causation (1)
• Strength of association
– “First upon my list I would put the strength of the association. To take a very
old example, by comparing the occupations of patients with scrotal cancer
with the occupations of patients presenting with other diseases, Percival Pott
[in 1775] could reach a correct conclusion because of the enormous increase
of scrotal cancer in the chimney sweeps. ‘Even as late as the second decade
of the twentieth century’, writes Richard Doll (1964), ‘the mortality of chimney
sweeps from scrotal cancer was some 200 times that of workers who were not
specially exposed to tar or mineral oils and in the 18th century the relative
difference is likely to have been much greater.’”
• Consistency
– “Next on my list of features to be specially considered I would place the
consistency of the observed association. Has it been repeatedly observed by
different persons, in different places, circumstances and times?”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”
Proceedings of the Royal Society of Medicine, 58 (1965), 295-300.

21. Bradford Hill Criteria for
Causation (2)
• Specificity
– “One reason, needless to say, is the specificity of the association, the third
characteristic which invariably we must consider. If as here, the association is
limited to specific workers and to particular sites and types of disease and
there is no association between the work and other modes of dying, then
clearly that is a strong argument in favor of causation.”
• Temporality
– “My fourth characteristic is the temporal relationship of the association –
which is the cart and which is the horse? This is a question which might be
particularly relevant with diseases of slow development. Does a particular diet
lead to disease or do the early stages of the disease lead to those particular
dietetic habits? Does a particular occupation or occupational environment
promote infection by the tubercle bacillus or are the men and women who
select that kind of work more liable to contract tuberculosis whatever the
environment – or, indeed, have they already contracted it? This temporal
problem may not arise often, but it certainly needs to be remembered,
particularly with selective factors at work in the industry.”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”

22. Bradford Hill Criteria for
Causation (3)
• Biological Gradient
– “Fifthly, if the association is one which can reveal a biological gradient,
or dose-response curve, then we should look most carefully for such
evidence. For instance, the fact that the death rate from cancer of the
lung rises linearly with the number of cigarettes smoked daily, adds a
very great deal to the simpler evidence that cigarette smokers have a
higher death rate than non-smokers. The comparison would be
weakened, though not necessarily destroyed, if it depended upon, say,
a much heavier death rate in light smokers and a lower rate in heavier
smokers. We should then need to envisage some much more complex
relationship to satisfy the cause and effect hypothesis. The clear dose-
response curve admits of a simple explanation and obviously puts the
case in a clearer light.”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”

23. Bradford Hill Criteria for Causation (4)
• Plausibility
– “It will be helpful if the causation we suspect is biologically plausible. But this
is a feature I am convinced we cannot demand. What is biologically plausible
depends upon the biological knowledge of the day.”
• Coherence
– “On the other hand, the cause-and-effect interpretation of our data should
not seriously conflict with the generally known facts of the natural history and
biology of the disease – in the expression of the Advisory Committee to the
Surgeon-General, it should have coherence.”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”

24. Bradford Hill Criteria for
Causation (5)
• Experiment
– “Occasionally it is possible to appeal to experimental, or semi-
experimental, evidence. For example, because of an observed
association some preventive action is taken. Does it in fact prevent?
The dust in the workshop is reduced, lubricating oils are changed,
persons stop smoking cigarettes. Is the frequency of the associated
events affected? Here the strongest support for the causation
hypothesis may be revealed.”
• Analogy
– “In some circumstances it would be fair to judge by analogy. With the
effects of thalidomide and rubella before us we would surely be ready
to accept slighter but similar evidence with another drug or another
viral disease in pregnancy.”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”

25. Bradford Hill Criteria for
Causation (6)
• Tests of Significance
– “No formal tests of significance can answer those
questions. Such tests can, and should, remind us of the
effects that the play of chance can create, and they will
instruct us in the likely magnitude of those effects. Beyond
that they contribute nothing to the ‘proof’ of our
hypothesis.”
Austin Bradford Hill, “The Environment and Disease: Association or Causation?”
Proceedings of the Royal Society of Medicine, 58 (1965), 295-300.

26. Summary List
1. Strength (effect size): A small association does not mean that there is not a causal effect, though
the larger the association, the more likely that it is causal.
2. Consistency (reproducibility): Consistent findings observed by different persons in different
places with different samples strengthens the likelihood of an effect.
3. Specificity: Causation is likely if there is a very specific population at a specific site and disease
with no other likely explanation. The more specific an association between a factor and an effect is,
the bigger the probability of a causal relationship.
4. Temporality: The effect has to occur after the cause (and if there is an expected delay between the
cause and expected effect, then the effect must occur after that delay).
5. Biological gradient: Greater exposure should generally lead to greater incidence of the effect.
However, in some cases, the mere presence of the factor can trigger the effect. In other cases, an
inverse proportion is observed: greater exposure leads to lower incidence.
6. Plausibility: A plausible mechanism between cause and effect is helpful (but Hill noted that
knowledge of the mechanism is limited by current knowledge).
7. Coherence: Coherence between epidemiological and laboratory findings increases the likelihood
of an effect. However, Hill noted that "... lack of such [laboratory] evidence cannot nullify the
epidemiological effect on associations".
8. Experiment: "Occasionally it is possible to appeal to experimental evidence".
9. Analogy: The effect of similar factors may be considered.

27. Descriptions and Coding of Adverse
Events
• Coding of reported adverse events to a
standardized set of terms from the various
terms used by investigators (“verbatim
terms”) is necessary for analysis, especially
when combining trials.
– Coding, as well as inaccurate verbatim
descriptions can, however, introduce problems
when looking for a safety signal.

28. Descriptions and Coding of Adverse
Events
• Classification of verbatim term from
investigator to preferred term may result in:
– Lumping dissimilar events
• e.g. “Fell and fractured arm” and “Got dizzy and fell
fracturing arm” both coded to “accidental injury”
– Splitting similar terms
• e.g. “vision blurred”, “visual acuity reduced”, and
“accommodation disorder” may all reflect the same
symptom and in aggregate be easily seen as a signal,
but individually not
Modified from presentation by Judy Racoosin MD, MPH Senior policy
advisor in the FDA Office of the Center Director, CDER. April 2009,
Philadelphia.

29. NDA Integrated Summary of
Safety
• 21 CFR 314.50(d)(5)(vi) A summary and
updates of safety information, as follows:
– (a) The applicant shall submit an integrated
summary of all available information about the
safety of the drug product, including pertinent
animal data, demonstrated or potential adverse
effects of the drug, clinically significant drug/drug
interactions, and other safety considerations, such
as data from epidemiological studies of related
drugs.

30. Safety reporting in clinical
summaries
• Avoid ambiguous or sweeping terms such as “generally,”
“apparently,” “slight,” or “comparable”; at least without
qualification.
• Avoid inappropriate lumping; e.g., “overall, AEs ‘comparable’,”
despite knowing certain ones weren’t;
or splitting; e.g., “while abdominal pain was more common on
drug, stomachache was more common on placebo.”
• Report relative AEs based on explicit, consistent, and
meaningful rules; for example, if justified by sample size and
distribution of AE rates: “The following AEs were reported at
a frequency of at least 3% on active and at a rate at least
twice that of placebo”:
– Headache (10%, 4%; respectively); nausea (7%, 3.5%);
insomnia (4%, 1.5%), rash (3%, 1%).
* 

31. One last word on study
summaries
• Remember that the purpose of any study
summary is not to show that a drug is safe or
is effective, but to report and assess results
accurately and objectively. Clinical study
summaries and integrated summaries are not
commercial documents.

32. Safety Assessment During Drug
Development
• Goals of FDA Safety Review
– Critically examine sponsor’s contention drug is safe for
intended use
• Assess adequacy of testing for safety
• Identify any safety issues that impact approvability
– Identify safety issues that should be described in product
labeling
Modified from presentation by Judy Racoosin MD, MPH Senior policy advisor;
FDA Office of the Center Director, CDER. April 2009, Philadelphia.

33. FDA Review of Safety Data in
NDA
• “We generally do not give much credence to
the investigator’s attributions.”
– “The conclusions proffered by the investigator
tend to correlate with the expected AEs described
in the Investigator’s Brochure.”
– “Novel AEs that are not traditionally thought of as
drug-related are not likely to be identified as such
(e.g. tendon rupture with fluorquinolones).”
Modified from Judy Racoosin MD, MPH Senior policy advisor;
FDA Office of the Center Director, CDER. April 2009, Philadelphia.

34. Limitations of safety assessment in clinical trials
for predicting safety after approval
• Sample size inadequate to detect infrequent AEs
– Statistical tidbit: “The rule of 3”
• The rule of three gives a quick way to estimate the kinds of
probability that “something hasn’t happened yet.” It says that if
you’ve tested N cases and haven’t found an event, a reasonable
estimate of the probability of that event occurring in that
population is less than 3/N.
– For example, if 3000 subjects have been studied without an event,
the probability of that event occurring (in the larger population) is
< 3/3000 (0.1%).
• That’s nice… but let’s say that probability is actually 0.09%
– The probability of its occurrence in the first 5,000 patients who take
the drug is about 99% (with an expected number of 4.5).

35. Limitations of safety assessment in clinical trials
for predicting safety after approval (2)
• Event may require longer duration of use to become manifest
• Reporting and observation are usually biased, especially when
relying on “spontaneous” reporting
• Populations at risk not adequately represented
– Co-morbid conditions controlled or excluded
– Clinical trial populations healthier than patients with same disease in
the general population
– Concomitant medications controlled or excluded (potential drug-drug
interactions not fully assessed)
– Real-life confounders, e.g. non-compliance, not factored in
– Limited numbers of population subsets (elderly, non-white, etc.)
Modified from Judy Racoosin MD, MPH Senior policy advisor;
FDA Office of the Center Director, CDER. April 2009, Philadelphia.

36. FDA Guidance for Industry Premarketing
Risk Assessment (March 2005)
• “Even large clinical development programs
cannot reasonably be expected to identify all
risks associated with a product.
• “Therefore, it is expected that, even for a
product that is rigorously tested pre-approval,
some risks will become apparent only after
approval, when the product is used in tens of
thousands or even millions of patients in the
general population.”

37. FDA Guidance for Industry
Premarketing Risk Assessment
• “The fewer a product’s demonstrated
benefits, the less acceptable may be higher
levels of demonstrated risks. Likewise, the
fewer the benefits, the less uncertainty may
be accepted about a product’s risks.”

38. Withdrawn drugs (post-approval) 2001-2013

39. • Subject safety always comes first, and safety assessment must be
thorough, thoughtful, and truthful.
• Signals of important adverse drug reactions may not be detected,
especially in early trials: vigilance throughout and beyond development
is required.
– Uncommon drug-related AEs are unlikely to appear in relatively small
clinical trials
– Common non-drug related AEs are likely to appear and may obscure a true
drug-related signal
– People tend to see what they want or expect to see, and not to see what
they don’t expect or desire: investigators and sponsors must remain open
to the unexpected and objectively assess what is observed
Summary & Final Words

40. • Overall consideration of the balance of evidence in light
of factors suggesting “causation” is critical for
determining significance of a “signal” in clinical trials.
– Overall consideration of the product’s “risks given its benefits” is
essential for appropriate clinical use (& approval)
• May need to do targeted clinical safety study trial or
post-approval studies to get more information
• In any study report or summary, safety data must be
presented objectively, with no attempt to obfuscate or
minimize the possible significance
Summary & Final Words