The document discusses the evolution of statistical computing from a chaotic 'wild west' environment to a controlled framework using Statistical Controlled Environment (SCE) software, emphasizing the importance of metadata in improving project management and process efficiency. It highlights how SCE can provide transparency, traceability, and automated reporting of data analysis processes, addressing the challenges of manual systems. Ultimately, the document advocates for a shift towards metadata-driven solutions for better scalability and compliance with regulatory standards.

1. Case in Point: The Point of a Statistical Controlled Environment and a Case Study or Alice vs. Billy the Kid - How the Metadata Wonderland Trumps the Wild Wild Legacy System Daniel Boisvert

2. Outline The history of the wild, wild west* *The evolution of traditional Statistical Computing The Point What is an SCE* and how can it tame the west? *Statistical Controlled Environment The Case Study What does our SCE metadata wonderland look like now? Quality Project Management and Process Improvement Define.xml Metadata

3. There was a time when Statistical Computing was Free & Open

4. Wild Wild West Statistical Computing Framework Raw Data SDTM/ ADAM TLF Pgms. Final TLFs Delivery Dataset Pgms.

5. Wild Wild West Gaps SDTM Target AdaM specs TLF specs SAP Protocol Make files Multiple deliverables Raw Data SDTM/ ADAM TLF Pgms. Final TLFs Delivery Dataset Pgms. Draft TLF, Analysis Results Metadata Controlled Terminology, Value Level Metadata Define.xml

6. Point to Point Model Delivery Mgmt. TLF Specs QC Status File Share Manually Controlled by SOPs COTS ADAM Specs Draft Output Standard Macros Protocol,SAP, Final TLF EDMS Raw Data CDMS

7. Breakdowns Lead to Point Solutions Analysis Pgms. Standard Macros Make Files COTS Home Grown Software File Share Manually Controlled by SOPs VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec SDTM Target CDISC Metadata(VLM, ARM,CT) Protocol,SAP, Final TLF EDMS Raw Data CDMS

8. What is the Effect of a Change? Analysis Pgms. Standard Macros Make Files COTS Home Grown Software File Share Manually Controlled by SOPs VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec CHANGE SDTM Target CDISC Metadata(VLM, ARM,CT) Protocol,SAP, Final TLF EDMS Raw Data CDMS

9. Wild Wild West Workflow Programmer Makes Update Updates Spec Programmer reruns all TLF “just to be sure” Analysis Pgms. VSS Does this affect last weeks deliverable?

10. To anyone who can find out The Effect of the Change

11. What am I looking at? Version Control System / EDMS implemented Software programming workflow controlled Output is always QC’d Gaps: Delivery to Medical Writing is a manual transfer Output on server may be updated and complete, but not necessarily output in EDMS

12. Why change Now? Transparency between systems is non-existent Communication between systems is manual Not Scalable: The solution-centric model requires new systems for new problems Anything that falls outside the scope of a point solution can only be controlled manually

13. Ok, but do we have to? Increased responsibility for transparency FDA focus is Data “ In God we trust, all else send data” FDA reviewers want to verify results presented in NDA, and assess robustness/sensitivity of the results. CDISC define.xml Metadata : Clear mapping between the plans for analysis, the tabulation data and its values, the analysis data, and the analyses performed No longer possible manually

14. WabanSCE Analysis Pgms. SDTM Target CDISC Metadata(VLM, ARM,CT) Standard Macros Make Files VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec Raw Data CDMS Protocol,SAP, Final TLF EDMS Metadata Wonderland

15. II. The Point What is an SCE? The environment in which all transactions between CDMS and CSR take place Backboned by software that: Consolidates and connects previously disparate systems Automatically captures workflow metadata (Audit Trail) Allows various reports of the metadata

16. Metadata Wonderland What Metadata does SCE collect? Every who, what, when, where & how from CDMS to CSR Every handoff Every change that occurs with a reason for change Every connection between these data points

17. Metadata Wonderland For our typical process Upstream/Downstream dependencies Input(s) to programs (datasets, excel files, specifications) Output(s) they create Title and Table number Deliverables it was/will be included in State of the hierarchy when delivered Who created/modified each file Why it was created/modified Project Status QC status (When , Who, Pass/Fail/Draft/Final) How many outputs are remaining?

18. III. The Case How Genzyme capitalizes on this metadata Quality Project Management and Process Improvement Define.xml Metadata

19. Quality Out of the Box 21 CFR Part 11 Compliance Controlled program execution For every output all inputs are linked to the system Reproducibility is guaranteed Traceability from raw data to output Automatic reporting of “Stale” outputs Forced “correct” workflow Extended to Medical Writing Automatic Impact Analysis

20. Wonderland Workflow CHANGE Automatic: ADSF goes “Stale” 1 table go “Stale” System knows what to rerun WabanSCE Metadata ADAM Spec Automatically Stale: ADSF 14.1.1.3

21. Impact Analysis

22. Project Management & Process Improvement Transparency into Project status Appropriate resources as necessary Gather Metrics = Process Improvement Date of all ADAM complete/QC’d Date of all TLF complete/QC’d Number of changes to specs and reason for change How many hours does it take? Where are the bottlenecks? Create feasible timelines

23. Biostats missed the timelines! Response to Upper Management There were X changes requested post-dbl These changes were received only X days prior to deadline There were X programming errors that were caught in QC

24. Biostats self reflection Query SCE Metadata Use Query Results How do we meet more aggressive Timelines? Improvment

25. Analysis Results Metadata Clear mapping between the plans for analysis, the tabulation data, the analysis data, and the analyses performed

26. SCE Metadata table captures: Output Number Output Titles/Footnotes Specification References SCE Metadata links Output name to its Data Reference Define.xml becomes a view of SCE metadata Analysis Results Metadata

27. IV. Conclusion The Wild Wild West model is not scalable Implementation of metadata driven SCE software fills gaps and connects disparate systems Metadata reporting leads to greater transparency and increased efficiency

28. Questions I’ll take questions now & You can contact me with any further questions Dan Boisvert www.DanielBoisvert.com [email_address] (617) 768-6061