Case In Point

1,140 views

Published on

Illustrating the point of a Statistical Controlled Environment and how Genzyme capitalizes on the metadata.

1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
1,140
On SlideShare
0
From Embeds
0
Number of Embeds
41
Actions
Shares
0
Downloads
0
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide
  • Replace this slide with a slide that shows the handoffs and the room for error that is created by the handoffs.
  • Show workflow from silo to silo and how it is all people making the manual connections
  • Our process works and works well, but the bar has been raised and we need to step up.
  • Clear mapping between the plans for analysis, the tabulation data, the analysis data, and the analyses performed
  • Hub and spoke model, one stop shopping
  • Case In Point

    1. 1. Case in Point: The Point of a Statistical Controlled Environment and a Case Study or Alice vs. Billy the Kid - How the Metadata Wonderland Trumps the Wild Wild Legacy System Daniel Boisvert
    2. 2. Outline <ul><li>The history of the wild, wild west* </li></ul><ul><ul><li>*The evolution of traditional Statistical Computing </li></ul></ul><ul><li>The Point </li></ul><ul><ul><li>What is an SCE* and how can it tame the west? </li></ul></ul><ul><ul><li>*Statistical Controlled Environment </li></ul></ul><ul><li>The Case Study </li></ul><ul><ul><li>What does our SCE metadata wonderland look like now? </li></ul></ul><ul><ul><ul><li>Quality </li></ul></ul></ul><ul><ul><ul><li>Project Management and Process Improvement </li></ul></ul></ul><ul><ul><ul><li>Define.xml Metadata </li></ul></ul></ul>
    3. 3. There was a time when Statistical Computing was Free & Open
    4. 4. Wild Wild West Statistical Computing Framework Raw Data SDTM/ ADAM TLF Pgms. Final TLFs Delivery Dataset Pgms.
    5. 5. Wild Wild West Gaps SDTM Target AdaM specs TLF specs SAP Protocol Make files Multiple deliverables Raw Data SDTM/ ADAM TLF Pgms. Final TLFs Delivery Dataset Pgms. Draft TLF, Analysis Results Metadata Controlled Terminology, Value Level Metadata Define.xml
    6. 6. Point to Point Model Delivery Mgmt. TLF Specs QC Status File Share Manually Controlled by SOPs COTS ADAM Specs Draft Output Standard Macros Protocol,SAP, Final TLF EDMS Raw Data CDMS
    7. 7. Breakdowns Lead to Point Solutions Analysis Pgms. Standard Macros Make Files COTS Home Grown Software File Share Manually Controlled by SOPs VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec SDTM Target CDISC Metadata(VLM, ARM,CT) Protocol,SAP, Final TLF EDMS Raw Data CDMS
    8. 8. What is the Effect of a Change? Analysis Pgms. Standard Macros Make Files COTS Home Grown Software File Share Manually Controlled by SOPs VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec CHANGE SDTM Target CDISC Metadata(VLM, ARM,CT) Protocol,SAP, Final TLF EDMS Raw Data CDMS
    9. 9. Wild Wild West Workflow Programmer Makes Update Updates Spec Programmer reruns all TLF “just to be sure” Analysis Pgms. VSS Does this affect last weeks deliverable?
    10. 10. To anyone who can find out The Effect of the Change
    11. 11. What am I looking at? <ul><li>Version Control System / EDMS implemented </li></ul><ul><ul><li>Software programming workflow controlled </li></ul></ul><ul><ul><li>Output is always QC’d </li></ul></ul><ul><li>Gaps: </li></ul><ul><ul><li>Delivery to Medical Writing is a manual transfer </li></ul></ul><ul><ul><li>Output on server may be updated and complete, but not necessarily output in EDMS </li></ul></ul>
    12. 12. Why change Now? <ul><li>Transparency between systems is non-existent </li></ul><ul><li>Communication between systems is manual </li></ul><ul><li>Not Scalable: The solution-centric model requires new systems for new problems </li></ul><ul><li>Anything that falls outside the scope of a point solution can only be controlled manually </li></ul>
    13. 13. Ok, but do we have to? <ul><li>Increased responsibility for transparency </li></ul><ul><ul><li>FDA focus is Data </li></ul></ul><ul><ul><ul><li>“ In God we trust, all else send data” </li></ul></ul></ul><ul><ul><ul><li>FDA reviewers want to verify results presented in NDA, and assess robustness/sensitivity of the results. </li></ul></ul></ul><ul><ul><li>CDISC </li></ul></ul><ul><ul><ul><li>define.xml Metadata : Clear mapping between the plans for analysis, the tabulation data and its values, the analysis data, and the analyses performed </li></ul></ul></ul><ul><ul><ul><li>No longer possible manually </li></ul></ul></ul>
    14. 14. WabanSCE Analysis Pgms. SDTM Target CDISC Metadata(VLM, ARM,CT) Standard Macros Make Files VSS Draft Output Delivery Mgmt. QC Status TLF Spec ADAM Spec Raw Data CDMS Protocol,SAP, Final TLF EDMS Metadata Wonderland
    15. 15. II. The Point <ul><li>What is an SCE? </li></ul><ul><ul><li>The environment in which all transactions between CDMS and CSR take place </li></ul></ul><ul><ul><li>Backboned by software that: </li></ul></ul><ul><ul><ul><li>Consolidates and connects previously disparate systems </li></ul></ul></ul><ul><ul><ul><li>Automatically captures workflow metadata (Audit Trail) </li></ul></ul></ul><ul><ul><ul><li>Allows various reports of the metadata </li></ul></ul></ul>
    16. 16. Metadata Wonderland <ul><li>What Metadata does SCE collect? </li></ul><ul><ul><ul><li>Every who, what, when, where & how from CDMS to CSR </li></ul></ul></ul><ul><ul><ul><li>Every handoff </li></ul></ul></ul><ul><ul><ul><li>Every change that occurs with a reason for change </li></ul></ul></ul><ul><ul><ul><li>Every connection between these data points </li></ul></ul></ul>
    17. 17. Metadata Wonderland <ul><li>For our typical process </li></ul><ul><li>Upstream/Downstream dependencies </li></ul><ul><ul><li>Input(s) to programs (datasets, excel files, specifications) </li></ul></ul><ul><ul><li>Output(s) they create </li></ul></ul><ul><li>Title and Table number </li></ul><ul><li>Deliverables it was/will be included in </li></ul><ul><li>State of the hierarchy when delivered </li></ul><ul><li>Who created/modified each file </li></ul><ul><li>Why it was created/modified </li></ul><ul><li>Project Status </li></ul><ul><ul><li>QC status (When , Who, Pass/Fail/Draft/Final) </li></ul></ul><ul><ul><li>How many outputs are remaining? </li></ul></ul>
    18. 18. III. The Case <ul><li>How Genzyme capitalizes on this metadata </li></ul><ul><li>Quality </li></ul><ul><li>Project Management and Process Improvement </li></ul><ul><li>Define.xml Metadata </li></ul>
    19. 19. Quality <ul><li>Out of the Box 21 CFR Part 11 Compliance </li></ul><ul><li>Controlled program execution </li></ul><ul><ul><li>For every output all inputs are linked to the system </li></ul></ul><ul><ul><li>Reproducibility is guaranteed </li></ul></ul><ul><ul><li>Traceability from raw data to output </li></ul></ul><ul><li>Automatic reporting of “Stale” outputs </li></ul><ul><ul><li>Forced “correct” workflow </li></ul></ul><ul><ul><li>Extended to Medical Writing </li></ul></ul><ul><ul><li>Automatic Impact Analysis </li></ul></ul>
    20. 20. Wonderland Workflow CHANGE Automatic: ADSF goes “Stale” 1 table go “Stale” System knows what to rerun WabanSCE Metadata ADAM Spec Automatically Stale: ADSF 14.1.1.3
    21. 21. Impact Analysis
    22. 22. Project Management & Process Improvement <ul><li>Transparency into Project status </li></ul><ul><ul><li>Appropriate resources as necessary </li></ul></ul><ul><ul><ul><li>Gather Metrics = Process Improvement </li></ul></ul></ul><ul><ul><ul><ul><li>Date of all ADAM complete/QC’d </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Date of all TLF complete/QC’d </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Number of changes to specs and reason for change </li></ul></ul></ul></ul><ul><ul><ul><ul><li>How many hours does it take? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Where are the bottlenecks? </li></ul></ul></ul></ul><ul><ul><ul><li>Create feasible timelines </li></ul></ul></ul>
    23. 23. Biostats missed the timelines! <ul><li>Response to Upper Management </li></ul><ul><ul><li>There were X changes requested post-dbl </li></ul></ul><ul><ul><li>These changes were received only X days prior to deadline </li></ul></ul><ul><ul><li>There were X programming errors that were caught in QC </li></ul></ul>
    24. 24. Biostats self reflection Query SCE Metadata Use Query Results How do we meet more aggressive Timelines? Improvment
    25. 25. Analysis Results Metadata <ul><li>Clear mapping between the plans for analysis, the tabulation data, the analysis data, and the analyses performed </li></ul>
    26. 26. <ul><li>SCE Metadata table captures: </li></ul><ul><ul><ul><li>Output Number </li></ul></ul></ul><ul><ul><ul><li>Output Titles/Footnotes </li></ul></ul></ul><ul><ul><ul><li>Specification References </li></ul></ul></ul><ul><li>SCE Metadata links Output name to its Data Reference </li></ul><ul><li>Define.xml becomes a view of SCE metadata </li></ul>Analysis Results Metadata
    27. 27. IV. Conclusion <ul><li>The Wild Wild West model is not scalable </li></ul><ul><li>Implementation of metadata driven SCE software fills gaps and connects disparate systems </li></ul><ul><li>Metadata reporting leads to greater transparency and increased efficiency </li></ul>
    28. 28. Questions <ul><li>I’ll take questions now </li></ul><ul><li>& </li></ul><ul><li>You can contact me with any further questions </li></ul><ul><li>Dan Boisvert </li></ul><ul><li>www.DanielBoisvert.com </li></ul><ul><li>[email_address] </li></ul><ul><li>(617) 768-6061 </li></ul>

    ×