This document summarizes key topics in clinical trials and the impact of regulations. It discusses the proliferation of multinational trials and varying interpretations of regulatory guidelines. Globalization has increased the number of countries and sites involved in trials. Regulations have increased costs without necessarily improving trials. The guidelines focus monitoring on critical events rather than exhaustive validation. New strategies aim to streamline processes while maintaining scientific rigor.

1. Clinical trials and the impactClinical trials and the impact
of regulationsof regulationsof regulationsof regulations
Cytel ConferenceCytel Conference
October 12, 2012October 12, 2012Octobe , 0Octobe , 0
David L DeMets, PhD
Department of Biostatistics &Department of Biostatistics &
Medical Informatics
U i it f Wi i M diUniversity of Wisconsin-Madison

2. TopicsTopics
• Proliferation of multinational trials
• Regulatory Guidelines & their• Regulatory Guidelines & their
interpretation
Ad t ti f f t• Adverse event reporting for safety
• One site monitoring
• SOPs & SAPs
• DMCsDMCs
• Adaptive Designs
N G id li f Di b t D• New Guidelines for Diabetes Drug
Approval

3. Regulatory GuidelinesRegulatory GuidelinesRegulatory GuidelinesRegulatory Guidelines
• Over the past 20 years, there are more
large phase III trials done by industry
• As a result, there are more regulatory, g y
guidelines on the design, conduct and
analysis of clinical trials (Eg ICH, FDA,y ( g , ,
EMA)
• Are the recent regulatory guidelines onAre the recent regulatory guidelines on
the design, conduct and analysis of
industry sponsored trials making trialsindustry sponsored trials making trials
better?

4. OpinionOpinion
• Clinical trials are not necessarily better
today than 20 years agotoday than 20 years ago
• The guidelines are generally consistent
ith li i l t i l f d t lwith clinical trial fundamentals
• The problem is in the interpretation of
those guidelines and practice
• Industries have been built up on thep
interpretation and implementation of
regulatory guidelinesg y g
• Not getting a better bang for the cost!

5. Globalization of InvestigatorsGlobalization of Investigators
Percent of Total
1572s Filed
Average Number of CountriesAverage Number of Countries
Where Clinical Trials Conducted
(2010)
Phase I Studies 2
Phase II Studies 13
20011997 2005 2009
Phase III Studies 34
5
Source: Tufts CSDD
20011997 2005 2009

6. Adult NonAdult Non--US TrialsUS Trials
• More than doubled in previous 10 years
• One-third of clinical trials by 20 largestOne third of clinical trials by 20 largest
US-based companies are conducted
solely outside the USsolely outside the US
• US Inspector General Report
80% of drugs approved had non US sites– 80% of drugs approved had non-US sites
– 80% of subjects from non-US sites
T i l f d lit ll– Trials of good quality generally
R f Gli k t l NEJM 2009• Ref: Glickman et al, NEJM 2009

7. US SituationUS Situation
• Sponsors often view US sites asp
– Too slow to get studies approved
– Too expensivep
– Too slow to enroll
• US investigators often view trials as• US investigators often view trials as
– Competing with pressures to see patients
Regulatory overhead more costly than– Regulatory overhead more costly than
actually taking care of the patient
Not enough academic currency– Not enough academic currency

8. Estimated Costs of DrugEstimated Costs of DrugEstimated Costs of DrugEstimated Costs of Drug
Development ($Millions)Development ($Millions)
Fee R: The cost of clinical trials. Drug Discovery and Development Webcast. March 1, 2007
DiMasi JA et al: The price of innovation: new estimates of drug development costs. J Health Economics 2003; 22:151-185p g p ;
DiMasi JA: New drug development in the United States from 1963 to 1999. Clin Pharmacol Ther 2001; 69:286-96

9. Multiple Countries / RegulatoryMultiple Countries / RegulatoryMultiple Countries / RegulatoryMultiple Countries / Regulatory
Agencies / regulationsAgencies / regulations
• Sponsors want their produce approved
in multiple countries – especially largep p y g
markets
• Want to abide by each country’sWant to abide by each country s
regulations for approval
• Not all countries have same regulations• Not all countries have same regulations
• Sponsors feel the need to meet lowest
d i tcommon denominator

10. International Conference onInternational Conference on
Harmonization (ICH)Harmonization (ICH)
Att t t t d i l 1990’ t k• Attempt started in early 1990’s to make
regulatory guidance between US and
E i t t l ithEurope more consistent, also with
Japan
• Many guidelines developed on design,
conduct, analysis, reporting for
submissions
• Guidelines generally support goodg y pp g
principles (E.g. ICH-E9 Statistical )

11. Regulatory Side BusinessRegulatory Side Business
• Many companies (CROs) have been
developed to help sponsors &developed to help sponsors &
investigators interpret the various
regulations avoid problems & passregulations, avoid problems & pass
audits
Similarly companies developed to• Similarly, companies developed to
conduct audits for regulatory
compliancecompliance
• Sometimes the same company
• All of this activity adds substantial cost

12. Clinical Trials TransformationClinical Trials Transformation
I iti ti (CTTI)I iti ti (CTTI)Initiative (CTTI)Initiative (CTTI)
• Collaborative effort between US FDA,Collaborative effort between US FDA,
EMEA, academia and drug/device
industryindustry
• Centered at Duke University
Governed by an Executive Committee• Governed by an Executive Committee,
& operated by a Steering Committee
Id ifi d i h• Identified some practices that were
costly, perhaps not effective, and areas
f li ifor streamlining

13. CTTI ProjectsCTTI Projects
Two examplesTwo examplesTwo examplesTwo examples
• SAE Reports
Copies from a trial shipped to all participating sites– Copies from a trial shipped to all participating sites
– Not unblinded & so not useful to sites
– Not required by US federal regulations
A l di i i f i– A costly tradition, not very informative
– Use DMC process, either internal or external
– FDA has new IND safety reporting guideliney p g g
• CRF Complete Auditing
– Line by line on site auditing by CRA’s
Very costly to sponsor and investigators– Very costly to sponsor and investigators
– Two natural experiments suggest errors in NIH
(academic) type approach are small
GUSTO• GUSTO
• Breast Cancer Trial / Montreal Site Fraud

14. Clinical SafetyClinical Safetyyy
• Many possible AEs and SAEs
• Some can be prespecified, e.g.
– LFTs
– QT Interval
• Unexpected events challengingUnexpected events challenging
• Rare events challenging
S d t d l b k• Some adverse events are deal breakers
• Multiple comparison problem
• Short term vs long term surveillance

15. Coronary Drug ProjectCoronary Drug Project
Mortality ResultsMortality ResultsMortality ResultsMortality Results
Life table cumulative mortality ratesLife-table cumulative mortality rates,
Coronary Drug Research Project Group

16. Coronary Drug ProjectCoronary Drug Project
“ f“ f“Unreliability of small numbers”“Unreliability of small numbers”
Placebo Superior
Clofibrate Superior
z values for clofibrate-placebo differences in proportion of deaths
by calendar month since beginning of studyby calendar month since beginning of study
(Month 0 = March 1966, Month 100 = July 1974)

17. AE Standard Coding SystemsAE Standard Coding SystemsAE Standard Coding SystemsAE Standard Coding Systems
• AE’s often collected from patient
complaints, text recorded & coded
• Several adverse event coding systemsg y
• Often organized by body systems
• Subcategorized into events as reported• Subcategorized into events as reported
by investigator/patient
S f t itt ft i th i• Safety committees often review these in
tabular form by treatment arms
• (AE listings not helpful after awhile)

19. AE Standard Coding SystemsAE Standard Coding Systemsg yg y
• Generally these tables not helpful
At l l t l d t bl fill d• At one level, too granular and tables filled
with small number of events – difficult to
interpretinterpret
• At higher level, too many critical and non
critical events pooled together – also difficult
to interpret
• DMCs often need to select critical events and
create new safety variable(s) best not to becreate new safety variable(s), best not to be
done post hoc
• Need to sharpen our safety focus on AdverseNeed to sharpen our safety focus on Adverse
Events of Special Interest (AESIs)

20. CTTI/FDA AE ReportingCTTI/FDA AE Reporting
I iti tiI iti tiInitiativeInitiative
• As a result, FDA has issued updatedAs a result, FDA has issued updated
guidelines on AE reporting: IND Rule
• Focus on AE’s that matter• Focus on AE s that matter
• However,
Gl b l l t i t– Global regulatory agencies may not agree
– Tradition is hard to break
– Many jobs depend on the current practice
– What trial wants to be the “first” ?
• Industry struggling to deal with new rule

21. Site MonitoringSite MonitoringSite MonitoringSite Monitoring
• Regulations & good practice suggest
that CTs need to establish that trial had
– Real patients
– Relevant disease
– No allocation bias / randomization
– Intervention applied without bias
– Unbiased Outcome ascertainment
– All relevant outcomes reported
– Consent forms signedConsent forms signed

22. Site Monitoring:Site Monitoring:
C t P tiC t P tiCurrent PracticeCurrent Practice
• Need for site performance validationp
has led to industry practice of
– On site monitoring, multiple timesg, p
– 100% Case Report Form (CRF) line by line
validation
– Costly to both sponsor and sites
– Perhaps as much as 30-35% of trial costp
• CTTI On Site Monitoring Survey
– Industry/CROs: > 80% of trials– Industry/CROs: > 80% of trials
– Academic networks: < 30% of trials

23. Is 100% CRF validation costIs 100% CRF validation cost
ff ti ?ff ti ?effective?effective?
• Two natural experimentsTwo natural experiments
– GUSTO-I trial of tpa in cardiac patients
• NEJM, 1993NEJM, 1993
– Breast cancer trial
• NEJM, 1995,
• Some errors detected but no influence
or even noticeable affect on analysisor even noticeable affect on analysis
• But public often expects perfection but
we can’t afford itwe can t afford it
– Need to minimize bias

24. CTTI & Site MonitoringCTTI & Site Monitoringgg
• CTTI Recommendations
– Focus on what matters
– Develop on-line, not post-hoc, qualityp , p , q y
management
– Assess quality in key parametersq y y p
• Improve training of investigators
– 1000 sites with 1 patient vs 10 sites each1000 sites with 1 patient vs 10 sites each
with 100 patients?
• Share experiences work together toShare experiences, work together to
improve QC without adding burden

25. Multiple IRB ReviewMultiple IRB Reviewpp
• Multi-center trials will typically have
lti l IRB imultiple IRB reviews
• Multi-country trials will have multiple
IRBs operating under different
“regulatory” rules or guidelines
• Each protocol may get different review
& requests for different changesq g
• Every amendment must go through the
process across all IRBsprocess across all IRBs
• Need to promote more central IRBs

26. Standard OperatingStandard Operating
Procedures (SOPs) ManiaProcedures (SOPs) ManiaProcedures (SOPs) ManiaProcedures (SOPs) Mania
• Good practice to document certain keyGood practice to document certain key
aspects for a trial
– DefinitionsDefinitions
– Procedures & equipment
– Training / certification– Training / certification
– Data collection
• Current practice is to document• Current practice is to document
everything imaginable, whether critical
or not it’s now a businessor not – it s now a business
• Audit teams demand your SOPs

27. Statistical Analysis Plans (SAPs)Statistical Analysis Plans (SAPs)
ManiaManiaManiaMania
• Many statistical methods exist for any
single question or datasingle question or data
• Multiple comparison, repeated testing,
endless subgroup issuesendless subgroup issues
• Good to write down before trial starts an
analysis plan for major question(s)
• BUT not all contingencies can be
anticipated and planned for
• Need to use statistics as a tool to helpNeed to use statistics as a tool to help
us think, not an excuse to stop thinking

28. Greenberg ReportGreenberg Report
Recommendations for CT ConductRecommendations for CT ConductRecommendations for CT ConductRecommendations for CT Conduct
• NIH Report 1967, CCT 1988NIH Report 1967, CCT 1988
• Develop a mechanism to terminate early if
Question has been answered– Question has been answered
– Trial can’t achieve its goals
Unusual circumstances– Unusual circumstances
– Hypothesis no longer relevant
S h ld i i l• Sponsor should not terminate a trial
without outside consultants
• Led NIH to use of external DMCs

29. DMCs in IndustryDMCs in Industry
• Increased use of DMCs since 1990
• FDA 1989 guidelines very brief mention of dataFDA 1989 guidelines very brief mention of data
monitoring and DMCs
• International Conference on Harmonization
(ICH)
– ICH/E9
• Section 4.5 Interim Analyses
• Section 4.6 Independent DMCs
ICH/E6– ICH/E6
• Gene therapy patient death (2000)
29
• FDA DMC Guidelines (2001, 2005)

30. FDA: When External DMCs AreFDA: When External DMCs AreFDA: When External DMCs AreFDA: When External DMCs Are
Probably NeededProbably Needed
• Trials with mortality or major morbidity
endpointsp
• Trials for which assessment of serious
toxicity requires comparison of ratestoxicity requires comparison of rates
• Trials of novel, potentially high-risk
treatmentstreatments
• Trials with high risk or vulnerable subjects

31. DMC Guideline ImpactDMC Guideline ImpactDMC Guideline ImpactDMC Guideline Impact
• DMCs formed for Phase III and some
Phase II trials, more & more
• More DMCs than available experiencedMore DMCs than available experienced
members
• No sustained training programs for• No sustained training programs for
DMCs
Despite books and numerous papers– Despite books and numerous papers
– No regulatory science training
FDA till ibl f d t• FDA still responsible for adequate
DMCs implementation

32. Industry-Modified NIH Model
(PROMISE 1991 NEJM)
Pharmaceutical Industry
Sponsor
Steering
Committee
Regulatory
Agencies
(PROMISE, 1991, NEJM)
Sponsor
Independent
Data Monitoring
Committee Agencies
Central Units
Data Monitoring
Committee (IDMC)
Statistical Data Central Units
(Labs, …)
Data Coordinating
Center (Sponsor or
CRO)
Statistical Data
Analysis Center
Clinical Centers
Institutional
Review Board
Patients

33. Statistical Data AnalysisStatistical Data Analysis
Center (SDAC)Center (SDAC)
• FDA DMC guidelines suggest a
preference for an SDAC independent of
the sponsor (industry or NIH)
• Not enough such centers
• Most CROs not experienced at DMC
supportsupport
– DMC Reports often not focused or
presented effectivelyp y
– Often lack flexibility via contracting
mechanism

34. DMC ChartersDMC ChartersDMC ChartersDMC Charters
• Early charters were written to provide
guidance to DMC
– Membership
– Responsibilities
– Guidelines for early terminationy
• FDA guidance supports this approach
• Recent charters becoming more• Recent charters becoming more
contractual, suggest legal liability
DMCs expected to live by the charter– DMCs expected to live by the charter
– Rules rather than guidelines

35. DMC SAPsDMC SAPs
• Difficult to develop a total SAP for the
final analysis of a trial
• Impossible to develop an SAP for thep p
DMC of a trial
– Too many unexpected scenariosy p
– “Expect the unexpected”
– Attempts to have detailed DMC SAP haveAttempts to have detailed DMC SAP have
failed
• Plan for primary outcome a leadingPlan for primary outcome, a leading
secondary outcome, let DMC do it’s job

36. Indemnification of DMCsIndemnification of DMCs
• DMCs or members have been
subpoenaed and become defendants inp
litigation.
– Experience indicates investors most likely
• DMCs must be indemnified by the sponsor
or through some other defined process
• Indemnification language should be part of
the DMC Charter as well as contracts
• Should not be compromised by the
insertion of “negligence” or similar terms
as an exclusion for protection.
• Ref: DeMets, Fleming et al (2004, CCT)

37. Erosion of DMC IndependenceErosion of DMC IndependenceErosion of DMC IndependenceErosion of DMC Independence
• Charters read like a legal document, not aCharters read like a legal document, not a
set of principles to protect patients
• Charters may limit the number of interim• Charters may limit the number of interim
analyses, or endpoint data to be reviewed
(don’t spend any alpha!)(don t spend any alpha!)
• SDACs not always experienced in DMCs
T i i f DMC b l ki• Training of DMC members lacking
• Threat of litigation against DMCs may
influence their judgment negatively

38. DMCs and Adaptive DesignsDMCs and Adaptive Designs
• Adaptive designs popular but not new
Adaptation of sample si e based on• Adaptation of sample size based on
interim results still problematic
St ti ti l th d i t t t l T I– Statistical methods exist to control Type I
– Logistics still not fully resolved
• DMCs being asked to make sample size
adjustment recommendations
• May not be a good idea
– DMCs know totality of datay
– Sample size adjustment may be contrary

39. Some Common MythsSome Common MythsSome Common MythsSome Common Myths
• DMCs should be blindedDMCs should be blinded
• DMCs must follow the SAP exactly
• DMC meetings must be scheduled• DMC meetings must be scheduled
precisely & limited in number
• DMC reports must be on cleaned• DMC reports must be on cleaned
adjudicated data
• DMC reports can be totally• DMC reports can be totally
preprogrammed – following SAP
• DMCs review each AE or SAE
39
• DMCs review each AE or SAE

40. New Guidelines for DiabetesNew Guidelines for Diabetes
D A lD A lDrug ApprovalDrug Approval
• Recent guidelines require a ruling out ofRecent guidelines require a ruling out of
cardiovascular risk for drug approval
• For initial approval rule out risk of > 1 8• For initial approval, rule out risk of > 1.8
• Must follow with further data to rule out
i k f 1 3a risk of > 1.3
• OK if there are two consecutive trials
• Problematic if done within a single trial
– Filing “1.8 results” NDA may compromiseFiling 1.8 results NDA may compromise
trial continuation to get “1.3 results”

41. SummarySummary
• RCTs our best methodology if correctly
designed, conducted & analyzed
• Unnecessary burdens growing with narrow
regulatory interpretation
• Burdens increasing costs for sponsors &
investigators dramatically
• US role diminishing• US role diminishing
• Must streamline and focus more on real
critical issuescritical issues
• If we fail, other external forces will play a
major role & not be as well informed
• In the end, our patients lose