Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Flaskdata.io automated monitoring for clinical trials

107 views

Published on

In the race to deliver a COVID-19 vaccine, technology can be used to automate patient safety monitoring and assure that patients and physicians have valid data in order to make good decisions regarding risks and benefits.

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

Flaskdata.io automated monitoring for clinical trials

  1. 1. Danny Lieberman, Founder and CEO Clinical Trial Monitoring in a COVID-19 era The language of automation for clinical trials June 2020 flaskdata.io
  2. 2. • To understand how automation can be used to speed delivery of valid data to decision makers in clinical trials, we must first understand where we are today and how we got there. • Today, virtual trials are popular because of COVID-19. • We’ll show that virtual trials are complex distributed systems with a new set of problems. • We’ll review the history of clinical trials and show that little has changed since the big streptomycin trial in 1948 • We’ll devote the last part of the talk to how monitoring automation works. This talk
  3. 3. The language of automation • Collect - collect data from patients, investigators and devices. The study designers should be able to choose the right model for their therapeutic and patient population. Whether fully virtual or site-centric or a hybrid - the study designers should have reliable tools to collect data. Data collection can be from a phone, a desktop, a Web browser or an API. It should not matter. • Detect - automated detection of protocol violations uses real-time data and user behavior to detect anomalies and validate data streams. • Act - automated detection should be able to trigger an automated response; whether a push notification to a site coordinator phone or a Web hook callback.
  4. 4. 1 Act Detect Collect 2 3 Flask API
  5. 5. 3 Virtual trials A brief history We're not in Kansas anymore Monitoring distributed systems 4 Outline 1 2 3 4
  6. 6. Virtual trials Virtual Clinical Trials (VCTs), also called remote or decentralized trials, are a relatively new and yet underutilized method of conducting clinical research taking full advantage of technologies such as apps, electronic monitoring devices, and online social engagement platforms. https://leoinnovationlab.com/2020/04/02/virtual-clinical-trials-create-new-possibilities-for-patients-
  7. 7. Virtual trials: Complex distributed systems Virtual clinical trials may save time and money. John Robert Zibert - LEO Innovation Lab 4/2020 eCOA ePRO eSource eConsent EDC
  8. 8. 3 Virtual trials A brief history We're not in Kansas anymore Monitoring distributed systems 41 2 3 4
  9. 9. A brief history of time. Daniel 1 526BCE Avicenna Canon of Medicine James Lind and the scurvy trial 1025 1747 Streptomycin RCT 1946 FDA Guidance on RBM 2013
  10. 10. Daniel 1 Please test your servants for ten days: Give us nothing but vegetables to eat and water to drink.  Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see.”  So he agreed to this and tested them for ten days. At the end of the ten days they looked healthier and better nourished than any of the young men who ate the royal food.  So the guard took away their choice food and the wine they were to drink and gave them vegetables instead.
  11. 11. Avicenna - Drug testing rules ‣ Disease without complications ‣ 2 contrary cases ‣ At time of action and reproducibility
  12. 12. James Lind and Scurvy trial ‣ Cohort 1 Gruel and mutton-broth ‣ Cohort 2 Quart of cyder a day ‣ 50 years later British Navy made lime compulsory
  13. 13. Clinical trial monitoring basics (1948) ‣ Communicate with PI and site staff ‣ Review site processes, procedures ‣ Verify data accuracy
  14. 14. ICH E6 and ISO 14155:2011 ‣ Monitoring plan ‣ Design, complexity, size, and endpoints ‣ Consistent with FDA Guidance
  15. 15. FDA Guidance for RBM (2013) ‣ Centralized monitoring can replace onsite visits (1) ‣ Focus on site, not patient oversight ‣ Response: 2-3 days to 5-7 weeks after events ‣ Mitigate safety, quality risk ‣ Too slow for virtual/hybrid trials in a COVID-19 era (1) Bakobaki et al. The Potential for Central Monitoring Techniques to Replace On-Site Monitoring:(2012)
  16. 16. Over-monitoring ‣ Monitoring is non-invasive and seems harmless (1) ‣ 72–99% of ICU alarms clinically insignificant ‣ Cry wolf syndrome (1) Feder and Funk, Over-monitoring and alarm fatigue: For home the bells toll? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4926996/
  17. 17. Onsite monitoring with CRAs ‣ Onsite monitoring 20-35% of study cost ‣ Log deviations 7-13 weeks after data event ‣ SDV updates < 3% of data ‣ Creates data cleaning bottleneck at study close
  18. 18. Monitoring with Dropbox (2020) ‣ Remote monitoring with PDF ‣ More paper ‣ Less site visits. Not good.
  19. 19. 3 A brief history We're not in Kansas anymore Monitoring distributed systems 41 2 3 4 Virtual trials
  20. 20. Regulatory and reimbursement conflicts Generalizable data from near-real-life usage Well-designed controlled clinical studies Security and privacy vulnerabilities
  21. 21. Vintage monitoring of virtual trials does not work Sites Study monitors Doing what they feel like Things happening fast Patients are anywhere 10x
  22. 22. Submission by December 17 Need to revisit clinical trial design and operations ‣ Wearables, devices, apps ‣ Cloud services and APIs ‣ Real-time data & response for real-life usage
  23. 23. 3 A brief history We're not in Kansas anymore 41 2 3 4 Virtual trials Monitoring distributed systems
  24. 24. Monitoring by alerts ‣ Alerts urgent, important, actionable and real ‣ Symptoms better than causes ‣ Visualize trends
  25. 25. What are alerts? ‣ A metric over / under a threshold ‣ GCP, safety, common sense ‣ Rate of data acquisition
  26. 26. In your monitoring plan ‣ Monitor for you ‣ Symptom-based ‣ Latency. Fast. Fast. Fast. 10x
  27. 27. Alerts require 4 things ‣ Detection ‣ Enable mitigation ‣ Share knowledge ‣ Route to decision maker
  28. 28. Writing alert rules ‣ Metric and threshold ‣ Over-monitoring harder than under-monitoring ‣ Classify: Data latency, Protocol violation, AE/SAE, Recruitment
  29. 29. Taking action ‣ Playbook is important ‣ Rule or family of rules ‣ What it means and how to mitigate
  30. 30. Alert V&V ‣ Validation: Are we calculating the right metric? ‣ Verification: Are we calculating the metric right?
  31. 31. Alerts, issues and email ‣ Don't send mail ‣ Issue-tracking systems may help ‣ Goal is simplicity and speed 10x
  32. 32. Tracking & accountability ‣ Track and periodically review alerts ‣ If WFM remove it ‣ If alert < 50% accurate its broken
  33. 33. Over-optimism ‣ Causes beneath noise ‣ Symptoms may arrive late ‣ Rules may be more complex than issues
  34. 34. Automated detection & response is now a requirement Wrap-up ‣ Patient-centric/virtual/hybrid clinical trials are complex distributed systems with new challenges. ‣ The old people and process methods used by CROs are too slow in a COVID-19 era ‣ Lessons can be learned from complex system monitoring
  35. 35. Thank you’s. ‣ Jenya, Rivka, Eugene, Alex, Oleg, Anat, Batya and Dan for joining me on the flaskdata.io journey ‣ Sergey and his DevOps soldiers for playing in the mud with us ‣ Tim and Gene for allowing me to join their adventure at Fidelis Security ‣ Rob Ewaschuk’s observations while a Site Reliability engineer at Google ‣ Lana’s insight on conflicts
  36. 36. To learn more dannyl@flaskdata.io twitter.com/flaskdata flaskdata.io

×