We are moving towards a digital age of automation in clinical trials where electronic management of content and data to document activities and decisions is the industry-wide norm. This means that the concept of TMF is also evolving and is no longer just a document repository, but rather a collection of systems holding content and data. As a result, there is a strong business case from moving away from just archiving documents to actively managing TMF content and information using techniques and practices borrowed from clinical data management. In this webinar, we will look at how you can use queries, data review and process-based approaches to ensure that your TMF tells an accurate and complete story of what occurred in your study. We will also discuss the greater scope of TMF relevant information and systems and how you can best integrate these sources into a more holistic environment to greatly facilitate inspections and clinical trial oversight.

2. Transforming eTMF
Management –
Moving to a Data
Driven Approach
Paul Fenton, CEO,
Montrium
2

3. 3
House
Keeping
LIVE Webinar
• This webinar is being recorded and
will be made available after this
session
• We will send these slides to your
email at the end of the webinar

4. • Founded in 2005
• Working Exclusively in the Life Sciences
• Headquartered in Montreal, Canada
• EU HQ in Brussels
• Clients in North America, Europe & Asia
• Leading Content Management Platform
• Over 8000 Users in 20+ Countries
• Experienced Professional Services Group
4
About
Montrium
Connecting People,
Processes &
Technology
About the company

5. 5
Meet Your Speaker
Paul Fenton
President and CEO,
Montrium

6. 6
Electronic Content
Management & Business
Intelligence tool for
Clinical Trials
connect management platform for the life sciences
Regulatory Document
Management &
Submission Planning tool
for Drugs and Devices
Integrated and Data
Driven Quality
Management tool

7. We have evolved…however…
7
Things are changing

8. Key Clinical Data Management Techniques
• Data collection planning and definition based on study design
• Edit checks and query management
• Data interchange
• Risk based data review
• Use of electronic forms for data collection

9. The next
generation of
eTMF will be
data driven
• Scope of eTMF is expanding beyond documents and to other
systems
• Metadata and audit trail have become key to telling the story of
the eTMF and the clinical trial
• TMFs still tend to be siloed which makes it difficult to reconstruct
study activities and events
• We still don’t know if we have all of the information we need to
tell a comprehensive story of the study
• We need to take a different approach as data plays a more
important role in our eTMFs

10. We should leverage
techniques from clinical
data management
10

11. Expansion of TMF
scope
11

12. TMF Relevant
Systems
Clinical
Data
Operational
Data
IRT
EDC
PV
CS
PKPD
Labs
eTMF
cQMS
Img
RIM
CTMS
Legal
Clinical Data
Warehouse

13. 13
TMF relevant systems
• Many electronic systems are now used in clinical trial conduct
• These systems need to be either accessible to inspectors through links or copies
of the records they contain need to be filed in the eTMF
• This is not always practical
• We need to find better ways of grouping records from different systems in one
holistic place
• We still need to be able to show the relationships between these records which
can be challenging if they are in multiple systems

14. We need more
comprehensive
standards
14

15. Existing and future standards
15
TMF
Reference
Model
eTMF
Exchange
Mechanism
Standard
Standard
Clinical
Process
Model
(future)
Naming Conventions
Structure
Metadata
Content and Data Interchange
Study events, activities and processes
Timelines/Milestones
Process Mapping and Dependencies
Content clustering

16. Data collection
planning and
definition based on
study design
16

17. Predictive
completeness
• Completeness today is far from accurate
• To calculate accurate completeness we
need information
• By taking a process modeling approach to
TMF coupled with system integration we
can improve things
• Process modelling allows for the generation
of placeholders based on expected
processes, event and artifacts
• Processes are triggered from underlying
data sources
• Machine learning could be used to
continuously improve on this

18. 18
Signature
Pages
Protocol
Amendment
eCRF
Revision
SAP
Revision
IRB
Approvals
Statistical
Program
Amendment
Reg
Submissions
``
UK
FR
US
UK
01
UK
02
UK
03
FR
01 FR
02
FR
03
US
01
US
02
US
03

19. • Today’s eTMF systems organize
content based on the reference model
• It is difficult to reconstruct events
using the RM
• Clinical process models will allow
better visualizations of specific
processes
• Artifacts are clustered around these
models
• Processes are triggered from events
• Significantly improves the telling of the
story during inspections
19A B O U T T H E C O M P A N Y
Process-based
visualizations

20. Ⓒ Montrium 2018
Data Driven
Approach
Qualifying / Associated
Artifact(s) / Data
Artifact
Identification &
status
Files /
Datasets
Descriptive
Metadata
Source
Repositories
Processes
Placeholders
Event
Timelines
IRT EDC PVCS eTMFcQMSRIMCTMS

21. 21
Processed
Based
Visualization
- Example

22. Edit checks and query
management
22

23. Data driven
approach to
improve
quality
metrics
• Today we tend to focus more on filing
documents
• While the quality of the individual documents
are important, the eTMF should also be
logical in its content
• By applying data management type
techniques, similar to what we use in EDC, we
can run logical checks on our eTMF metadata
to ensure that it tells an accurate story
• This approach will allow us to gain a degree
of confidence in the quality of our TMF in
addition to the quality of the artifacts
themselves

24. • Anomalies could be detected based on predefined logical checks
• Logical checks would involve the verification of event sequence,
dates, versions etc. and would flag anomalies
• Anomalies would automatically raise queries
• Anomalies would contribute to risk scoring
• An eTMF health score could be generated to give an idea of the
logical health of an eTMF
TMF Health Metrics: Anomaly detection

25. Example Anomalies
Anomaly Rule Type Anomaly
Incorrect Event Sequence SAP Approval Date > Database Lock Date
Missing Dependency Statistical Analysis Complete event qualified with missing
Database Lock Complete event
Missing Mandatory Artifact Database specification artifact missing and Database ready
event qualified
Incorrect Milestone Sequence Ongoing Site MVR event qualified and current site milestone is
‘Startup’
Missing Metadata Missing IRB/IEC Approval Date for detected IRB/IEC Approval
Letter artifact
Missing Process Site IMP Reconciliation process is mandatory, however process
has not been qualified for specific site
Missing Version Events for Version 1.0. and version 3.0 of protocol exist however
no event for version 2.0. exists
Timeliness problem IRB/IEC Approval event occurs > 6 months after IRB/IEC
Submission event

26. • In data management we are able to generate automatic and
manual queries
• Automatic queries are either triggered on the fly as data is
added to the system (usually simple queries on one data
point) or in batch (usually more complex across multiple data
points)
• Manual queries are raised following manual review of data
• We could apply the same principles to eTMF which would
have a very positive impact on quality and the logical health
of the eTMF
Query Management

27. Data Interchange
27

28. • An extension of the TMF RM which focuses on the transfer
of content, metadata, audit trail and eSig information
• A TMF metadata standard
• A mechanism for exchanging TMF content between systems
• A method for describing TMF artifacts which is
comprehensible by both humans and machines
What is the eTMF-EMS
Electronic Trial Master File –
Exchange Mechanism Standard

29. • Current focus is on the transfer of documents and
associated metadata
• Scope could be extended to provide for more standard
metadata and the transfer of event related data
• This would facilitate the integration of not only eTMF
systems but other clinical systems that hold TMF relevant
information
• This would make identifying events much easier within the
clinical process model
Current and future scope of eTMF-EMS

30. Risk based review
30

31. The sponsor and/or investigator/institution
should implement risk-based quality checks (QC)
or review processes to ensure the TMF is being
maintained up-to-date and that all essential
documents are appropriately filed in the TMF.
- Guideline on the content, management and archiving of the clinical trial master file (paper and/or
electronic), Section 4.2., EMA, December 2018
31

32. Risk Scoring: The Ultimate Quality Metric
eTMF
Risk
Score
Timeliness/
Delays
Completeness/
Missing Artifacts
Artifacts
Linked to
Safety,
Efficacy or
Endpoints
Subject
Recruitment
Levels
Previous
Performance
(Site/Country)
Number and
Type of
Events
Previous
Inspection
Trends
Anomaly
rate
Rejection
Rate
32

33. Risk
modelling
and scoring
• Risk scoring would allow us to focus on high risk
areas which may be more prone to inspection
• Risk scoring over time would improve through
machine learning and by taking into account eTMF
inspection trends
• Risk scores could be calculated on individual
artifacts, processes, sites, countries, contributors
etc.
• Scores could be rolled up to create aggregate
scores to the clinical program level
• Risk metrics could be presented in an easy to
consume visual manner similar to completeness
metrics
• Risk scoring could not only help manage inspection
risk but also clinical risk in the future

34. Use of electronic forms
for data collection
34

35. Documents
and
becoming
data….
• Today the TMF is composed primarily
of documents, correspondence and
forms
• Forms tend to be ‘paper-like’ and we
do not exploit the data contained within
• There is a strong case for developing
electronic forms similar to what we use
in EDC
• We could make these forms available
through an EDC type interface and
data could either be imported or
entered directly into the forms

36. Documents
and
becoming
data….
• We could run edit checks to detect
issues and anomalies in real time
• This direct data capture approach
would mean that we could leverage
the data to better drive processes
• Additional forms would become
required based on predicted
processes
• We can also extract the information to
provide a much richer story of study
conduct

37. Benefits of
new
approaches
• By taking more of a processed based approach we
can improve our ability to generate accurate
completeness metrics and tell a better story
• By leveraging data management techniques we can
improve the quality of data contained within our TMF
• Risk scoring will allow us to focus our QC and review
efforts of higher risk areas rather than doing 100%
manual QC of document content which is
burdensome and does not necessarily produce
quality.
• By using interchange standards and technology we
can tell a more holistic story in one place
We need to get a better handle on our ability to tell an accurate story and a data driven
processed based approach will pave the way…

38. • A process based eTMF would be very dependent on quality and standardization
of underlying data – A significant amount of error handling would be required
• There could be many variances in processes and events based on study design
or region which could make the system configuration difficult to maintain – A
template based approach could help
• There would be significant bodies of data that would need to be analyzed which
would require significant computing power – Cloud computing could be the key
• Changes to underlying data sources could break overlying systems ability to
present data – Intelligent algorithms to detect errors due to underlying system
configuration changes and strong change management would help mitigate this
Challenges to Consider

39. Data Driven Approach is the
way to go….
39