The document outlines a metrics-driven approach to improve the management of clinical trials, focusing on establishing agreed metrics linked to company strategy, understanding performance control, and evaluating the impact of metrics on cost, quality, and time. It emphasizes the importance of training users, mapping metrics to processes, and effectively managing study-specific variations to enhance visibility and insight. Additionally, the document details a structured methodology for conducting site management and interim monitoring, including site visit processes and the development of metrics usage cases and visualizations for performance evaluation.

1. Making trials better, together
Adopting a Metrics Based
Approach for Running Clinical
Trials

2. 9
Goals
FOR A METRICS DRIVEN ORGANISAT

3. 1 – agreed
metrics, suppo
rting company
strategy
2 – understand
how to
control
performance
3 – understand cost, quality & time
impact of influencing a metric

4. 1 – agreed standard business
process
2 – use cases for all metrics
with supporting visualisations
3 – defined
process variances

5. 1 – users
informed and
trained on
process and
metrics
2 – metrics mapped
to process and
clinical systems
3 – study specific variances to
process managed

6. So, what
does achieving
those goals
give us?

7. Visibility

8. Insight

9. Control

10. That all sounds
great…

11. 5
Steps
A SIMPLE, PRAGMATIC, PROVEN APPR

12. ESTABLISH (CURRENT)
1 AND FUTURE STATE
PROCESS

13. 3. Site Management and Interim Monitoring 3.c. Site Monitoring
3.c.4 [ii] Conduct Interim Visit
Roles
CRA
CRA CRA CRA CRA CRA CRA
ACM
Conduct SDV and Update eCRF and/or CRF Tracking Complete ACM 3.c.4.17
3.c.4(i)
3.c.4.09 Notification Form 3.c.4.18
3.c.4.19
Review Supplies,
Process
Samples and If contact not on CTMS
Facilities
Track and Follow-up
Identify Review Deviations/ Liaise with Site Staff 3.c.4.13 On Site Work
New/Current Staff on Issues.
Deviations/ Violations Against as Significant and/or Completed for
Training Escalate as
Violations Escalation Criteria Urgent Issues Arise IMV
Re-evaluate 3.c.4.15 Necessary
3.c.4.10 3.c.4.11 3.c.4.12 3.c.4.20
Facilities Including 3.c.4.16(i)
Equip. Calibration
and Temp.
3.a.4 Monitoring i.5.1
3.c.4.14
Complete Checklist
Complete Checklist Questions Complete Checklist Questions
Questions Complete Follow-up
CTMS
Track Site and Subject Deviations/ Violations Complete Follow-up Items as Visit
Data
Complete Follow-up Items as Necessary
Track Deviation/Violation Source Data Review Necessary Completed
Items as Necessary Status and Action
Enter Any Additional Relevant Updates Link for any Deviations/ Date
Link for any Deviations/ Taken
Violations
Violations
Metrics
SOP
EDC via
Integration
System
CTMS CTMS CTMS CTMS CTMS CTMS
CTMS

14. AGREE METRICS AND
2 MAP
TO BUSINESS PROCESS
Three notes to keep in mind:

15. AGREE METRICS AND
2 MAP
TO BUSINESS PROCESS
Considerations as we select
the metrics:
• WHO WILL CONSUME THE METRICS?
• Ourselves?
• Our customers?
• Our shareholders?
• WHAT BEHAVIOURAL AND INFRASTRUCTURE CHANGES WILL
BE NEEDED?
• HOW EXACTLY WILL THE MEASUREMENT BE DERIVED?

16. AGREE METRICS AND
2 MAP
TO BUSINESS PROCESS
Considerations as we select
the metrics:
• WHAT WILL WE DO WITH THE RESULTS?
• Recovery actions
• Thresholds for action
• HOW DO WE PROVE THE DATA IS VALID?
• Source / mastering
• Timeliness
• ONGOING EVALUATION THAT WE HAVE THE RIGHT METRICS

17. Select Metric
Metric Title Category Metric Indicator
Metric # Type
21 Q Monitoring Visit Report Completion Monitoring LEADING Indicator
Definition (see Wiki for detailed definitions) Formula / Example Reporting Detail
Compliance with Monitoring Visit Report Completion Formula: X/Y (100) = % monitoring visits reports Study / Country / Region /
submitted for review within 14 calendar days of Therapeutic Area / Portfolio
site visit levels
X = # final monitoring visit reports submitted for
review (= iterations reviewed and subsequently
approved) within 14 calendar days of site visit
Y= total # reports
Example: 5675/6000*100 = 95%
Unit of Measure
%
Reporting
Business Driver(s) / Benefit Statement Additional Analysis on a "for cause" basis Target
Frequency
Compliance with monitoring metrics is directly linked to Analyze and act on trends during monthly Monthly > 95%
quality data, patient safety, and GCP (regulatory agency review.
compliance)

18. So, we
our list……
have

19. AGREE METRICS AND
2 MAP
TO BUSINESS PROCESS
Back to the
process maps…

20. 3. Site Management and Interim Monitoring 3.c. Site Monitoring
3.c.4 [ii] Conduct Interim Visit
Roles
CRA
CRA CRA CRA CRA CRA CRA
ACM
Conduct SDV and Update eCRF and/or CRF Tracking Complete ACM 3.c.4.17
3.c.4(i)
3.c.4.09 Notification Form 3.c.4.18
3.c.4.19
Review Supplies,
Process
Samples and If contact not on CTMS
Facilities
Track and Follow-up
Identify Review Deviations/ Liaise with Site Staff 3.c.4.13 On Site Work
New/Current Staff on Issues.
Deviations/ Violations Against as Significant and/or Completed for
Training Escalate as
Violations Escalation Criteria Urgent Issues Arise IMV
Re-evaluate 3.c.4.15 Necessary
3.c.4.10 3.c.4.11 3.c.4.12 3.c.4.20
Facilities Including 3.c.4.16(i)
Equip. Calibration
and Temp.
3.a.4 Monitoring i.5.1
3.c.4.14
Complete Checklist
Complete Checklist Questions Complete Checklist Questions
Questions Complete Follow-up
CTMS
Track Site and Subject Deviations/ Violations Complete Follow-up Items as Visit
Data
Complete Follow-up Items as Necessary
Track Deviation/Violation Source Data Review Necessary Completed
Items as Necessary Status and Action
Enter Any Additional Relevant Updates Link for any Deviations/ Date
Link for any Deviations/ Taken
Violations
Violations
Metrics
SOP
MCC21
EDC via
Integration
System
CTMS CTMS CTMS CTMS CTMS CTMS
CTMS

21. 3. Site Management and Interim Monitoring i.5. Site Visit Reports
i.5.1 Write Site Visit Report and Review
Roles
CRA CRA CRA CTM/OTL
2.b.2
Process
2.f.3
Generate Review Report for
Assign Report to Submit Report for
Completed Visit Completeness and
Reviewer Review i.5.2
Report For Review Consistency
i.5.1.2 i.5.1.3
i.5.1.1 i.5.1.4
3.c.4
Unit Tracking
No. of monitors on site
4.a.3 No. of hours spent on site (by attendee)
No of days spent on site by attendee)
Report Data
Date of Submission Comments
CTMS
Data
Unit Tracking Data List of Possible Name of Reviewer Date of Review
Reviewers Report Status = Homepage Alerts for LClinical
Report Status = In ‘Submitted to Reviewer’ (Reports Still Due for Review)
Progress
Metric
Metrics
SOP
Time from Visit
Completed to Initial
Submission
System
CTMS CTMS CTMS CTMS

22. 3 BUILD METRIC USE
CASES

23. What metrics are important to us?
How do I want to see performance
shown?

24. Where are our problem areas?
What is the root cause of the
problems?

25. What do I need to do to resolve
issues?
What can I do to prevent issues in
the future?

26. IDENTIFY DATA SOURCES
4 AND DESIGN
VISUALISATIONS
Data and sources:

27. IDENTIFY DATA SOURCES
4 AND DESIGN
VISUALISATIONS
Define data reposit
• Build for metric
• Data warehouse?
• ETL / web service

28. IDENTIFY DATA SOURCES
4 AND DESIGN
VISUALISATIONS
Back to the use cases…..
• Tab layout
• Graph type
• Dimensions
• Drill downs
• Companion data
• Call to action

29. IMPLEMENTATION AND
5 CHANGE MANAGEMENT

30. IMPLEMENTATION AND
5 CHANGE MANAGEMENT
The
Proce
ss
The What are the
Roles key ingredients
The for change?
How
The
Why

31. Dashboard Demo……
31

32. Want to know more?
Ben McGraw Duncan Hall
By Phone: +1 617 7021602 By Phone: +44 1279 812550
By E-mail: By E-mail:
mds@tibco.com duncan.hall@triumphconsultancy.co.
k