The document discusses operational challenges in executing adaptive trials and how the ACES system addresses them. ACES provides a secure centralized storage system called the DMC Portal to prevent operational bias. It loads pre-tested analysis programs into its ACES Engine to generate interim reports on blinded data. ACES also establishes trustworthiness through a non-invasive audit trail that timestamps who accessed what data and documents.

1. Execu&on of Adap&ve Trials:
Opera&onal Considera&ons
Cytel User Group Mee&ng
Paris, 14 October, 2011

2. Part II: Operational Considerations
• Sta&s&cal methodology for controlling type‐1
error is well established (Cui, Hung and Wang,
1999, East‐SurvAdapt, 2011)
• Logis&cal and opera&onal issues associated
with trial execu&on are less well established
• We have implemented a web solu&on for
handling the two main issues of concern to
regulatory agencies: opera&onal bias and
trustworthiness
2 Cytel UGM, Paris. 14 Oct 2011

3. What is Required?
"A well‐trusted firewall established for trial
conduct beyond those established for
conven&onal group sequen&al trials can help
provide assurance that sta&s&cal and
opera&onal biases have not been introduced."
FDA Guidance on Adap/ve Design (2010)
3 Cytel UGM, Paris. 14 Oct 2011

4. Operational Bias and
Trustworthiness
• Opera&onal Bias
– Can knowledge that the sample size was increased
(or not increased) affect the integrity of the
study?
• Trustworthiness
– Who saw what data, and when?
– Can a non‐invasive audit trail of the en&re data
handling process be implemented?
4 Cytel UGM, Paris. 14 Oct 2011

5. ACES: Access Control Execution
System
• Web based technology to control flow of
informa&on and access to confiden&al
documents
• Prevents opera&onal bias by including the
actual adap&ve algorithm only in DMC charter
and tracking access to this document
• Establishes trustworthiness through secure
password protected access to documents,
execu&on of algorithms, and audit trail
5 Cytel UGM, Paris. 14 Oct 2011

6. Traditional Process
6
Create
Documents
(Protocol,
SAP, DMC
Charter)
Store/Archive
Documents
Enroll Subjects
& Collect
Responses
Send
Response
Data to ISC
Perform
Analysis and
Create
Reports
Send Analysis
to DMC
Sponsor ISC DMC
Make
Recommenda&on
Send
Recommenda&on
to Sponsor/
Steering
Commibee
Make Decision
About Trial
Steering
Commibee
Acer decision…
1. DMC no&fied
2. Drug Supply no&fied
3. IVRS no&fied
Create and
Test Analysis
Programs
Cytel UGM, Paris. 14 Oct 2011

7. ACES Process
7
Create
Documents
(Protocol, SAP,
DMC Charter)
Store/Archive
Documents
Enroll
Subjects &
Collect
Responses
Send
Response
Data to ISC
Perform
Analysis and
Create Reports
Send Analysis
to DMC
Sponsor ISC DMC
Make
Recommenda&on
Send
Recommenda&on to
Sponsor/ Steering
Commibee
Make Decision
about Trial
Steering
Commibee
in ACES
Acer decision…
1. DMC no&fied
2. Drug Supply no&fied
3. IVRS no&fied
in ACES in ACES
Create and
Test
Analysis
Programs
Load Final
Analysis
Programs
into ACES
Cytel UGM, Paris. 14 Oct 2011

8. Sign In
8 Cytel UGM, Paris. 14 Oct 2011

9. Document Dashboard
9 Cytel UGM, Paris. 14 Oct 2011

10. Execution Cycle
10
Interim Analysis begins… Responses Generated… Reports Generated.
Cytel UGM, Paris. 14 Oct 2011

11. Audit Trial:
Document Access Log Summary
11 Cytel UGM, Paris. 14 Oct 2011

12. Review: What Problems did ACES
Address?
• DMC Portal
– secure centralized storage of documents
– customized access for DMC, ISC and Sponsor
• ACES Engine Generates Interim Reports
– analysis programs pre‐tested and loaded to ACES
– blinded dataset uploaded to ACES
• Non‐Invasive Audit Trail
– who sees what and when is &me stamped
– dataset and analysis program available for review
12 Cytel UGM, Paris. 14 Oct 2011

13. Concluding Remarks
• Adap&ve design reduces risk of failing to
detect a smaller clinically meaningful effect
• Pragma&c approach; wait to see data before
commikng addi&onal resources
• Sta&s&cal methodology well understood
• Opera&onal challenges and trustworthiness
are regulatory concerns that are handled well
by the ACES system
13 Cytel UGM, Paris. 14 Oct 2011