Presented at Foundry Health's eSource meeting in Chicago, 03-Oct-2017

1. practical tips on getting validated for go-live and taking updates
SaaS System Validation

2. Introduction
• Computerised System Validation
• eSource system change control
• eSource system selection
• On-site audit

3. Structure
• Validation
• Agile
• SaaS
• Combine!

4. Validation: Why Validate?
EMA’s Annex 11
‘Computerised Systems’
Principle
This annex applies to all forms of
computerised systems used as part
of a GMP regulated activities. […].
The application should be validated;
IT infrastructure should be qualified.
FDA’s 21 CFR Part 11
Subpart B—Electronic Records
§ 11.10 Controls for closed
systems.
Persons who use closed systems […]
shall employ procedures and
controls […]. Such procedures and
controls shall include the following:
(a) Validation of systems to ensure
accuracy, reliability, consistent
intended performance, and the
ability to discern invalid or altered
records.

5. Validation: Why Validate?
Business Impact
(extract from Gamp 5 guideline, section 1.5 ‘Business Benefits):
[…] Specific benefits to both regulated companies and suppliers include:
• early defect identification and resolution leading to reduced impact on cost and schedule
• cost effective operation and maintenance
• effective change management and continuous improvement
• providing frameworks for user/supplier co-operation
• assisting suppliers to produce required documentation
• promotion of common system life cycle, language, and terminology
• promoting pragmatic interpretation of regulations
• […]
GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008

6. Validation: What is System Validation
‘Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management ‘, ISPE, 2016

7. Validation: What is System Validation
Traditional validation procedures describe
• Project Phase or Initial system validation
• An Assessment (Regulatory Applicability, System Risk,
…)
• Requirements & Specifications
• Plan
• Protocols which demonstrate ’fitness for intended use’
• Matrix, matching requirements to tests
• Report
• System Operation phase:
• Change control
• Periodic Review
• Retirement phase:
• System retirement

8. Agile System Software Development
Product Backlog Sprint Backlog Sprint
Daily Standup
24h
2-3
weeks
Potentially
shippable
product
Iteration
Potential issue in the regulated industry

9. Agile System Software Development
Fast Train:
System Development
Slow Train:
System Validation, Training, ..
Validation of version 1.0
V2.0 Sprint 1 V2.0 Sprint 2 V2.0 Sprint 3
Validation of version 2.0
V3.0 Sprint 1 V3.0 Sprint 2 V3.0 Sprint 3
Version 1.0 in production!

10. Agile Testing Quadrants
• Testing is key in agile
development
• ‘Potentially shippable
product’
• Product should be
functionally tried and
tested when sprint is
finished
Agile Testing =
OQ?

11. SaaS – Software As A Service
Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’
What is SaaS and why is it confusing?
“Some of the following factors are still confusing the discussions in the
industry: the term “cloud” is used as though it is one homogeneous thing,
without consideration that SaaS, PaaS, and IaaS are very different, for
instance, and that deployment models vary. Also, some things described
informally as cloud are not really cloud, if you recognize generally
accepted definitions of the essential characteristics of cloud, and are really
just flavours of outsourcing.”

12. SaaS – Software As A Service
Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’
Example:
a consultancy firm resells software and hosts it on their own
servers in a collocation data center (Infrastructure as a
service, IAAS). An auditor enters the room, asks for the
coverage on backup & recovery for their service, and the
consultancy firm presents the ISO27001 or SOC certificate
from the datacentre they’ve used.

13. SaaS – Software As A Service
Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’
This certificate might be useful to mitigate risks related to:
• Physical security to the datacentre
• Business continuity aspects like internet lines
This certificate is completely irrelevant for ISO27001 aspects like:
• Backup and recovery: the datacentre doesn’t care if you backup your data
• Disaster recovery aspects: if that datacentre goes down; your servers too. It’s your duty to
get a contract with another data center & ensure sufficient failover mechanisms
• Business continuity aspects like server failure: they’re not the datacenter’s servers
• Validation aspects, quality of software & services, SLA, …

14. Working with SaaS providers which create
Software using an Agile methodology in a
Regulated industry

15. Regulated Company versus SaaS Provider
Extract from ‘Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management ‘, ISPE, 2016
1 Introduction
In the evolving regulated IT environment there are many things to consider when thinking of turning to the
cloud for a solution. […] While it is not universally true, SaaS providers delivering specialized support to
regulatory business processes (e.g., Clinical Trails, Release Testing, AE reporting) tend to have a good
understanding of the needs of regulated companies.
Using a SaaS provider can be an excellent option for regulated companies, but doing appropriate research and
identifying the company’s specific support needs are critical to making the right choice of SaaS provider. […]

16. Regulated Company versus SaaS Provider
Blog post of Sion Wyn (ISPE, Conformity LTD) : on ‘Cloud Computing Solutions & Providers Assessment & Management’
Approaches to assessment and management of technology service providers must be flexible, practical, and
pragmatic, and insisting on physical audits of all providers, regardless of type of service or level of risk, is
unrealistic.
The three key elements of regulated company management of technology service providers are:
• Appropriate risk assessments (taking into account the nature of the process, the data, and in the case of cloud-
based solutions, the service model and deployment model)
• Supplier/provider assessments of the primary provider and the proposed solution (including their management
of sub-suppliers)
• Agreements/Contracts/SLAs in place to establish the controls that are managed by the service provider
It is also unrealistic to insist that any service providers perform traditional and cumbersome paper-based
qualification activities, rather than encouraging them to apply effective IT good practices supported by
appropriate and modern tools and technologies.

17. Work together with the supplier
• Maximize supplier involvement, as encouraged by GAMP5:
2.1.5 Leveraging Supplier Involvement
Regulated companies should seek to maximize supplier involvement throughout the system life
cycle in order to leverage knowledge, experience, and documentation, subject to satisfactory
supplier assessment.
For example, the supplier may assist with requirements gathering, risk assessments, the
creation of functional and other specifications, system configuration, testing, support, and
maintenance.
Planning should determine how best to use supplier documentation, including existing test
documentation, to avoid wasted effort and duplication. Justification for the use of supplier
documentation should be provided by the satisfactory outcome of supplier assessments, which
may include supplier audits.
Documentation should be assessed for suitability, accuracy, and completeness. There should be
flexibility regarding acceptable format, structure, and documentation practices.
GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008

18. Work together with the supplier
GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008

19. User Involvement
We differentiate CSV Awareness & CSV participation:
• CSV Awareness is the client becoming aware of all procedures on validation, to be
able to:
• Stand and defend the computerized system during client audits or inspections
• Assess & even perform change control & periodic review
• CSV Participation is the client having an active role during the validation:
• Help Key Users, or Subject Matter Experts to understand applicable procedures
• Guide Key Users, or Subject Matter Experts to write scenario’s & execute test scripts
“Up-to-date software with up-to-date users”

20. Validation should not be seen as a big black box at the end of each
software implementation process, but should be structured in such a way
that parts of the validation can be performed during product setup,
configuration or even proof of concept phase.
CSV participation

21. CSV participation
Speed up validation performing activities throughout the onboarding process!

22. eSource is audit sensitive!
Auditors often have experience with eCRFs but not with eSource: source of confusion
eCRF system:
• Basic validation on system level
• Extensive testing on study level, eCRF solutions are often building boxes, tools to create forms, edit checks, …
eSource system:
• Extensive system level validation. E.g. Volunteer database requiring overall validation towards HIPAA, European directive
on data protection, interfaces to operational devices and external labs
• Study setup verification according to a well-defined SOP
A lot to explain, and as such a high need for proper validation documentation

23. Practical Tips
• IQ: Limit to Installation Verification
• OQ: leverage supplier activities for validation
GAMP5 Agile
Modules Epics
User Requirements User Stories
Functional Specifications Acceptance Criteria
• CQ: Limit to configuration verification
• PQ / UAT: Use key users to execute Scenario’s for each module
to check business processes

24. Conclusions
• Define internal validation SOP’s!
Cfr. GAMP5’s defined business benefits: ‘promote pragmatic
interpretation of rules’
• Validation effort helps to streamline your business processes
• Hire validation consultants with care!

25. Conclusions
• Leverage documentation of your SaaS vendor (without expecting
it to be in the same form as the regulated company)

26. Conclusions
• Perform on-site vendor audits to assess SDLC processes

27. Check applicable regulations!
• 1996: Health Insurance Portability and Accountability Act, HIPAA
• FDA:
• 2003: FDA’s 21 CFR Part 11 on Electronic Records / Electronic Signatures for US
• 2007: FDA: Guidance for Industry: Computerized Systems Used in Clinical Investigations.
• 2010: FDA: Guidance for Industry: Electronic Source Documentation in Clinical Investigations.
• 2011: FDA: Guidance for Industry: Oversight of Clinical Investigations – A Risk – Based Approach to Monitoring.
• ISPE:
• 2008: GAMP 5, A Risk – Base Approach to Compliant GxP Computerized Systems’, ISPE, 2008
• 2016: Using SaaS in a Regulated Environment – A Life Cycle Approach to Risk Management
• 2006: CDISC: Leveraging the CDISC Standards to Facilitate the use of Electronic Source Data within Clinical Trials à includes a
list of requirements for eSource systems.
• 2011: EMA’s Annex 11 on computerized systems for EU
• 2016: ICH / GCP for executing clinical studies: ICH HARMONISED TRIPARTITE GUIDELINE GUIDELINE FOR GOOD CLINICAL
PRACTICE E6(R1), step 4
• 2018: EU wide ‘Data Protection Directive’, streamlining processes for all eu countries