OHSUG 2012 presentation - a pragmatic approach to risk based monitoring


Published on

With the regulators moving forward with guidance on risk based monitoring, and the industry trend of
adopting this approach, this presentation will aim to demonstrate how the combination of CTMS, EDC
and analytics can be used to identify and manage site risk profiles. The presentation will draw from the
work completed by the MCC members and show how the Site Scoring tools for Site Selection and Study
Conduct can be used to generate a risk profile dashboard based on data from CTMS and EDC. The
presentation will further explore how data captured in CTMS and EDC can be used to update the risk
profile of the site during the course of the study, allowing the study team to proactively manage risk.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OHSUG 2012 presentation - a pragmatic approach to risk based monitoring

  1. 1. Tammy FinniganNov 06, 2012
  2. 2. ObjectivesThis presentation will aim to: • Provide a brief overview of the guidance that is driving the change in how the industry approaches monitoring • Define risk based monitoring • Examine the process and data required to generate risk profiles for investigational sites • Demonstrate how CTMS and EDC analytics can be used to identify and manage changing risks during study conduct • Consider the changing role of the monitor 2
  3. 3. Background Legislation The role of the monitor CFR and EU CTDir ICH- ICH-GCP “protect the safety and well being of “Sponsors of clinical human subjects and the quality and investigations are required to integrity of data” provide oversight to ensure Guidance from regulators: adequate protection of the “most effective way “ to monitor a clinical rights, welfare and safety of trial was to “maintain personal contact human subjects and the between the monitor and the investigator quality and integrity of the throughout the clinical investigation” data” interpretation: Industry interpretation Site qualification Regular monitoring visits (4-8wk intervals) 100% data verification 3
  4. 4. Change in industry guidance Recent Industry Guidance Growing consensus Re- Re-visited ICH-E6 ICH- Risk-based approaches to monitoring, focusing Flexibility in how trials are monitored on critical data elements, more likely to: Advice to sponsors: • ensure subject protection • overall study quality Consider “the objective, purpose, design, complexity, blinding, size and Will permit sponsors to monitor the conduct of endpoints of a trial” in determining the clinical trials more efficiently and effectively extent and nature of monitoring than the current approach of routine visits to all clinical sites and 100% data verification Guidance specifically provides for the possibility of reduced, or even no* Interesting point: onsite monitoring and address data Recent survey by an EDC vendor on >2000 quality and site performance through protocols demonstrated that < 3% of the clinical centralized methods study data was changed due to onsite *Guidance is clear only appropriate in monitoring exceptional circumstances 4
  5. 5. Risk Based Monitoring • Identify likely • Critical data sources of • Investigator • On-going • Processes error experience data analysis identified in the Protocol Study Site • Impact of error Monitoring risk • Historical Proactive • Identify non- Risk Risk • Likelihood of Plan assessment performance Risk Mgmt. compliance Assessment Assessment error • On site and • Standard of • Identify • Ability to centralized care in country outliers detect error activities Considerations Considerations Considerations Considerations • Type of data being • Complexity of study • Outcome of protocol risk • Aggregate data analysis collected design assessment and and review outlying sites • Specific activities • Types of end points monitoring plan • Analyse site required for data • Stage of study (taper • Site quality assessment characteristics and collection monitoring) • Increased areas of risk correlate with poor • Data critical to reliability • Standard of medical for the site performance and non- of study findings care • Additional area of risk compliance trends • Safety concerns • Type of monitoring for the site • Adjust monitoring • Subject protection activities (frequency, • Site specific monitoring activities based on the concerns intensity, targeted, plan analysis random, onsite/central) 5
  6. 6. Site Risk Profiles • Not within the context of a specific study • Based on historical performance data across studies Site risk • Set risk threshold for key performance indicators • On-going quality management and development of investigator and site relationship profile • Based on protocol risk assessment and protocol monitoring plan • Cross reference protocol risk assessment against Site risk profile Study site • Include outcome of site qualification visit as applicable risk profile • Identify areas of additional risk for the site • Modify the protocol monitoring plan to accommodate additional site specific risks • Study risks should always be addressed in the study site monitoring plan Study site • Additional focus or intensity of monitoring activities may be planned for individual sites to mitigate specific risks monitoring plan 6
  7. 7. Key Performance Indicators The site risk profile should provide a continuous picture of the site F Subject Safety Data Quality Performance - Protocol violations - Data query rate - Data reporting time - Screen failure rates - % of missing data - Query response time - Drop out rates - Protocol violations and - Critical site issues - Compliance with safety deviations - Issue resolution time and ethics reporting - Source data verification - Recruitment rates timelines issues - GCP training 7
  8. 8. Proactive Risk Management Improved site and study performance Non-study Study related related 8
  9. 9. Centralized vs. On Site Monitoring When is On Site Centralized Monitoring On Site Monitoring Monitoring Effective? Recent survey by an Inexperienced sites or site Solely data monitoring EDC vendor on >2000 staff i.e. new to clinical • Statistical analysis protocols demonstrated trials, new to therapy area that < 3% of the clinical • Exception reporting Sponsor has no/little study data was changed • Data violations previous experience with due to onsite monitoring • Missing data the site With advances in • Data aggregation Training remote data capture • Root cause analysis systems, safety/PVG Maintaining and improving systems and internet investigator relationships • Trend analysis access across the When physical verification globe, is onsite is required i.e. drug Feedback loop into monitoring effective? accountability protocol and site risk assessments 9
  10. 10. Study Related Risk ManagementDuring the study, off site, centralized monitoring activities and data analysiscan be used to identify risk: Aggregate Data Analyse Site Statistical Methods Monitor Data Quality Analysis Characteristics Protocol specific rules Missing data Aggregate data across study sites Data anomalies Parametric algorithms Complete trend analysis and Inconsistent data Higher frequency of errors (probability rules) determine ‘expectedness’ Review sites in relation to other Fraud and bias detection Potential protocol deviations Higher protocol violations sites i.e. protocol distribution Exception reporting Data outliers Identify outliers Delays in data reporting 10
  11. 11. Aggregate Data AnalysisReview site data in relation to other study sites:Protocol ViolationsPV bubble chart from study site data analysis dashboard to show protocol site distribution ofprotocol violations, highlight outlier sites (with hover over displaying # by category i.e. eligibility,informed consent, IMP)- In this example there are sites that display disproportionate number of violations compared with other study sites- Root cause analysis - Drill down to the violation type, are there specific risk factors for subject safety and/or data integrity? - Mitigation – additional site training visit 11
  12. 12. Aggregate Data AnalysisScreen Failure RatesSF bubble chart from study site data analysis dashboard shows protocol site distribution ofscreen failure rates, highlight outlier sites (hover over displaying # by reason i.e. withdrewconsent, abnormal labs)- Are there sites that display disproportionate number of screen failures compared with other study sites?- Root cause analysis - Drill down to the screen failure reasons, are there specific risk factors for subject safety? - What practices are the site following? - What mitigation needs to be put in place?- Are there high rates of specific screen failure reasons across sites and countires?- Root cause analysis - Feed back into the protocol risk assessment - Have we detected something unexpected? - Do we need to change our risk assessment and monitoring plans? 12
  13. 13. Aggregate Data AnalysisReview study data to identify trends:Protocol DropoutsEarly withdrawal scatter diagram from study subject data dashboard show sprotocol distributionof early withdrawal reasons, and rates by country- Trend analysis - Is this the identification of additional risks to subjects and/or study procedures - Assess against protocol risk assessment - Do we need to adjust our monitoring and quality plans - Do we need to conduct additional training - Is there a need for country specific amendments 13
  14. 14. Site CharacteristicsReview site data to identify sites with characteristics correlatedwith poor performance and non-compliance:Study Site Risk ProfileRadar diagram from study site risk dashboard, to show site risk characteristics in relation tostudy ‘normal’ rate- The site is compared with the protocol average for critical risk factors to determine if there are: • A higher number of data errors • A lower number of subject dropouts • Longer lag time entering data 14
  15. 15. Proactive Risk Management Determine appropriate mitigation, update to risk assessments Follow escalation process identified in monitoring plan Identify new risks (study and/or study site) Review aggregate data dashboards 15
  16. 16. Risk MitigationStudy levelUnexpected trends identified in the protocol data may result in the need tochange monitoring activities across the study e.g.• Data previously deemed to be non-critical may need to be monitored more closely• Trends in subjective end point analysis may result in more onsite monitoring activities• Trends in protocol violations may point to the need for additional training of site staff 16
  17. 17. Risk MitigationSite levelUnexpected site characteristics identified in the study site data may result inthe need to modify the monitoring activities for the individual site e.g. • More intense data monitoring if there is an inordinate number of data errors in the sub-set that is monitored for all sites • Targeted onsite monitoring visits • Tapered monitoring visits i.e. additional site visits in the early stages of a study if high number of eligibility violations or screen failures, address trainingThe continuous picture of the site should not be forgotten during the study • Site profile should be monitored for change • As this is based on cross study performance, the site may move above/below the identified thresholds • Study teams should be informed 17
  18. 18. Changing RolesThe role of the monitor as currently defined, is going to change.Consider the following model: Site Quality Manager Site Manager Data (Quality) Manager Create site risk profile Site relationship management Data monitoring and analysis Assess site against study risk Site training Root cause and trend analysis profile Create study site monitoring Onsite data verification Ongoing study risk management plan 18
  19. 19. Benefits of risk based monitoringReal time identification of risk will allow sponsors and CROs to:• Make informed decisions• Be proactive in managing potential impact to subject safety and data integrityOn-going management of site risk profiles will ultimatelyimprove site and study performance 19
  20. 20. Questions 20
  21. 21. Presenter’s Biography and Contact details Tammy Finnigan, COO, Triumph Consultancy Services tammy.finnigan@triumphconsultancy.co.uk Tammy’s entire career has been focused on clinical research, having worked in project management and clinical operations for 10 years, with both large Pharma and CRO businesses prior to joining Triumph. Her experience both in monitoring, and managing clinical trials made her a significant hire for Triumph in 2007. Tammy’s experience, passion and eye for quality saw her promoted to Head of EU Operations within her first year, and in 2011 she was appointed COO to take over global operations responsibility. 21