Rx for FDA Software Compliance

Parasoft Corporation © 2016 1
2016-09-26
Rx for FDA Software Compliance
August 2016

Your Presenters
Arthur Hicken is Chief Evangelist at Parasoft where
he has been involved in automating various software
development and testing practices for over 20 years.
He has worked on projects including cybersecurity,
database development, the software development
lifecycle, web publishing and monitoring, and
integration with legacy systems.
Follow him on Twitter @codecurmudgeon
Kelly Weyrauch has more than 30 years of software and
systems development experience, and 20 years of focus on
software processes and quality systems for medical devices.
As an independent consultant, he works with software and
systems creators to apply Agile concepts and Quality
Management System requirements to the unique context of
their development environment.
Follow him on LinkedIn @kellyweyrauch

Agenda
What the FDA expects
IEC 62304
Benefits of static analysis
How to integrate static analysis
How to reduce noise and stop wasting time

2016-09-26
Introduction to Medical Device Software
Regulations
Where do Static Analysis Tools fit in?
Kelly Weyrauch – Agile Quality Systems LLC

 Software as a Medical Device
 Software-only
 Part of a medical device system
 Software used to develop/manufacture/manage a device
 Development environments & tools
 Automated test systems
 Manufacturing systems
 …
 Quality Management System tools
 CAPA & Issue Tracking systems
 Complaint handling systems
 Document Control
 …
Scope of Software Regulations
(c) Agile Quality Systems 2016

 21 CFR 820
 Quality System Regulation
 820.30 Design Controls
 21 CFR 11
 Electronic Records; Electronic Signatures
 ISO 13485
 Medical devices— Quality management systems
 Section 7 – Product Realization
 IEC 62304
 Medical device software— Software life cycle processes
 ISO 14971
 Medical devices — Application of risk management to medical devices
Software Development Regulations and Standards

 GPSV
 General Principles of Software Validation (FDA)
 AAMI TIR45:2012
 Guidance on the Use of AGILE Practices in the Development of Medical Device
Software
 AAMI TIR36:2007
 Validation of software for regulated processes
 Many other, more specialized standards & guidance documents
Software Development Guidance

 FDA is responsible for protecting the public health by assuring the safety,
efficacy and security of human and veterinary drugs, biological products,
medical devices, our nation’s food supply, cosmetics, and products that
emit radiation.
 FDA is also responsible for advancing the public health by helping to
speed innovations that make medicines more effective, safer, and more
affordable and by helping the public get the accurate, science-based
information they need to use medicines and foods to maintain and
improve their health. …
 ... (http://www.fda.gov/AboutFDA/WhatWeDo/)
 Same things Industry wants!
 Industry & Regulatory should align on the same goals
 Requires effort to show they are aligned
FDA Mission Statement

 Satisfying regulatory expectations should be a
natural result of doing good work
 Should not be viewed as
 Adversarial
 Conflicting
 Bureaucratic
 Legalistic
 Impractical
 Offensive to your engineering sensibilities
 …
Industry & Regulatory Alignment

 21 CFR 820 mentions software only 3 times
 820.30(g) Design Controls: Design Validation
 “Design validation shall include software validation and risk analysis, where appropriate.” (No definition of what
“Software Validation” means.)
 820.70(i) Production & Process Controls: Automated Processes
 “When computers or automated data processing systems are used as part of production or the quality system, the
manufacturer shall validate computer software for its intended use according to an established protocol.”
 820.181(a) Device Master Record
 “Device specifications including … software specifications”
 Software “requirements” (expectations) come from Guidance
Documents
 General Principles of Software Validation (GPSV)
 Guidance for the Content of Premarket Submissions for Software Contained in Medical
Devices
 Others
FDA Software Regulations

 Regulators often not experienced with
software
 For software as a medical device, may bring bias of
hardware-development experience, especially with
assumptions of a linear lifecycle
 For software tools, may bring bias of
manufacturing systems
 So you should be the software expert
 Showing how your process satisfies regulations
FDA Software Regulations

 Explain what you do
 In Processes and Plans
 Demonstrate that you do it
 By producing adequate records
 “Demonstrate” may seem like non-value-
added burden
 Especially to those doing the work
 Necessary, so find ways to optimize the value/cost,
to avoid the perception of “burden”
Be In Control,
Show That You Are In Control

 …Is ALWAYS a lousy reason
 Too simple
 Offloads responsibility for owning our process
 Often a cover for another argument, usually one
that is not well thought out and probably
dysfunctional
 We own our process (and the FDA does say that)
 But regulators are stakeholders to be satisfied
 Satisfy ourselves that your process is good,
and the regulators will be satisfied
“Cuz the FDA Said So…”

 Build a good product
 Safe & Effective
 Have a good process for building that product
 Do good work
 With evidence of that good work
 Tell a good story that will convince anyone
 Why you think your product is good
 Why you think your process is good
 That you know what you are doing
Keys To Success

 Specifically created for
medical device software
 Though many elements
are foundational to any
robust software
development process
IEC 62304

 A framework – processes, activities and tasks
 Identifies requirements for
 What needs to be done
 What needs to be documented
 Software only
 Assumes software is part of a broader system that
defines higher-level activities for design inputs and
design (product) validation
IEC 62304 – What it is

 Does not prescribe how to meet the
requirements
 Not a “how to” with defined methods or practices
 Does not require a specific software life cycle
 Does not specify documents
 Says what to document, not where it must go.
 These decisions are left to you
IEC 62304 – What it isn’t

IEC 62304 Medical Device Software –
Software Life Cycle Processes
SYSTEM development ACTIVITIES (including RISK MANAGEMENT)
Customer needs
Customer needs
satisfied
7 Software RISK MANAGEMENT
8 Software configuration management
9 Software problem resolution
Activities outside the scope of this standard
5.2
Software
requirements
analysis
5.1
Software
development
planning
5.8
Software release
5.7
Software SYSTEM
testing
5.3
Software
ARCHITECTURAL
design
5.4
Software
detailed
design
5.6
Software integration
and integration
testing
5.5
Software UNIT
implementation

 Mentioned briefly in FDA General Principles of
Software Validation, 2002
 In 2000’s, driven largely by concerns over the
high failure rate in infusion pump systems &
software, FDA recognizes value in using Static
Analysis techniques to reduce software errors
 Often an “expectation” in submissions and quality
system inspections
Static Analysis

Software Unit Implementation
Customer needs
Customer needs
satisfied
5.2
Software
requirements
analysis
5.1
Software
development
planning
5.8
Software release
5.7
Software SYSTEM
testing
5.3
Software
ARCHITECTURAL
design
5.4
Software
detailed
design
5.6
and integration
testing
5.5
Software UNIT
implementation

 Implement software units – write the code
 Establish acceptance criteria for software units
 Coding Standards are a “should have” (defined in the Plan)
 Establish strategies, methods and procedures for verifying each
software unit
 Reviews
 Static Analysis
 Unit Tests
 Address this in the Verification Strategy section of the Plan
 Perform and document results of software unit verification
 According to the strategies and methods
 Does not require all software units to be tested
Software Unit Implementation

Software Integration and Integration Testing
Customer needs
Customer needs
satisfied
5.2
Software
requirements
analysis
5.1
Software
development
planning
5.8
Software release
5.7
Software SYSTEM
testing
5.3
Software
ARCHITECTURAL
design
5.4
Software
detailed
design
5.6
and integration
testing
5.5
Software UNIT
implementation

 Integrate the software units (build the software system)
 Verify that software units have been integrated (build was
successful)
 Review of build procedures, make-files, configurations, etc.
 Compiler checks, static analysis, etc.
 Test integrated software system performs as expected in
accordance with plan
 Functionality, behavior, performance, etc
 Regression tests
 Document test results
 Can be done as part of software system testing (see later slide)
 Capture problems (software problem resolution process)
Software Integration and Integration Testing

 Define in your Software Development
Procedure/Processes:
 Strategy for using Static Analysis as a part of your
overall verification strategy
 Requirements for how and when to use it
 As part of code (unit-level) verification
 As part of integration verification
Recommendation for use of Static Analysis

 Kelly Weyrauch
 Kelly@AgileQualitySystems.com
 763-688-0980
 www.AgileQualitySystems.com
 Connect with me on LinkedIn
Contact Information

Recalls due to Software Failure
http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedical
ProductsandTobacco/CDRH/CDRHReports/UCM308208.pdf

FDA on Static Analysis
3.1.2 “Software testing is one of many verification
activities intended to confirm that software
development output meets its input requirements.
Other verification activities include various static
and dynamic analyses, code and document
inspections, walkthroughs, and other techniques.”
5.2.4 “Source code should be evaluated to verify its
compliance with specified coding guidelines.”
General Principles of Software Validation; Final Guidance for Industry and FDA Staff

What Choice
 Prevent Defect Insertion
Quality System
Design Control
Risk Management
 Remove Defects by Detection
Code Review
Design Review
Testing: unit, integration, system, alpha, beta
Customer Dissatisfaction
Recall
 Roll the dice
John Murray, US FDA, March 2008

Fix or Prevent

Static analysis is more than detection
 Relationship of automated analysis
 Preventative static analysis
 Flow analysis
 Runtime error detection
 Uninitialized memory example
 Runtime will find it IF the test suite is thorough
 Flow analysis may find it depending on complexity
 Pattern to prevent: Initialize variables upon declaration
 Many standards like MISRA are focused on
prevention rather than detection

Static analysis – what it can do
 Identify defective code - runtime bugs
 Flag defect-prone code (possible bugs and
“gotchas”)
 Suggest defensive programming practices
 Monitor application-specific guidelines (e.g.
portability)
 Enable policy enforcement (security)
 Flag unmaintainable / poorly readable / “dialect”
code

Sample Static Analysis Policy
 Which rules run on all code
 Which rules run on certain conditions
 International, performance, etc
 What level of compliance is required
 When are suppressions acceptable
 Where does SA fit the process

Static Analysis Feedback Loop
Identify
Error
Isolate
Root Cause
Fix
New Rule
to Prevent
Monitor
 Code review
 Regression
 QA
 Field bugs

How to choose rules
 Based on why you’re using static analysis
 Study expected issues
 Analyze bug-tracking system
 Don’t just turn on rules because it’s a good
idea
 Pick few enough to use sustainably

Why MISRA for Medical?
 Coding Standards
 Well-defined
 Updated
 Flexible
 Deviation Strategy
 Auditable
 What else would you start with?

FDA/MISRA Alignment
FDA Guideline MISRA Capability
“Least burdensome approach” Lightweight and flexible
Company defines standards Proven standards pre-packaged
Work must be traceable Provides traceability methodology
Process must be auditable Defines auditable reports

Sample Rules for FDA Static Analysis
 Avoid accessing arrays out of bounds
 Avoid use before initialization
 Avoid null pointer dereferencing
 Avoid overflows due to [various causes]
 Avoid division by zero
 Ensure deallocation functions guarantee resource
freeing
 Do not use resources that have been freed
 Do not free resources using invalid pointers
 Do not abandon unreleased locks
 Do not use blocking functions while holding a lock
 Ensure resources are freed
 Do not abandon unreleased locks
 Properly terminate character strings
 Never return a reference to a local object

Dashboards Vs Process Intelligence
1970s Electrocardiogram
• Isolated data points
• No priority
• No warnings – just data
• No analysis
21st Century Monitor
• Multi-variate analysis
• Customizable
• Critical alerts
• Valuable data

Harnessing “Big” Data
 Aggregate data
 Correlate data
 Mine data
 Create
 Reports
 Dashboards
 Tasks
 Alerts
 Continuous testing/delivery/release

Execution /
CI Build
Parasoft DTP
RawObservations
Parasoft Development Testing Platform
Static Analysis
Engine
Testing and
Coverage
PHPMD
API
FindBugs
API
CheckStyle
API
Other 3rd
Party …
API
Desktop IDE
Web UI
External SystemRequirements
and Defects
Source Control
Workflow
(Task)
Intelligence
(Dashboards/
Reports)
Practice/
Domain Data
(REST API)
Prioritized Findings
Web UI
Process
Intelligence
Engine
Policy Management

Parasoft Development Testing Platform
Optimal
Routes
Historic
Data
Road
Construction
Traffic
Conditions
Map/Road
Data
Speed
Limits

Parasoft DTP
Process
Intelligence
Engine
PIE – Derivative (Predictive) Analysis
External Systems
Application
Hotspots
Prioritized
Actionable
Information
External
Processes
3rd Party System
CI/CD/CR
Infrastru
cture
System
Monitoring
Product
FeedbackCustomer
Support
Code
Review
Metrics
People
Code
Defect
Static
Analysis
Tests
User
Stories Coverage
Parasoft DTP

What can PIE do?
What do I test first?
Where is the
business logic and
how testable is it?
Test Stability Index;
Look at your
unstable tests.
Identify valuable
Tests
Find Application
Hotspots: Technical
Risk vs Business Risk,
Technical Debt,
Density KPIs,
Clusters
I made changes,
which tests need to
be re-run?
What is the risk of
new code? Did I test
the new code
changes?
Don’t make the
problem worse &
review existing
issues when your
working with
code

2016-09-26
Real world examples

Customer Challenge:
Reducing the Burden of Compliance
 In order to develop solutions for the
pharmaceutical market, a company must not
only follow very strict requirements, but also
prove that they actually satisfied these
rigorous expectations. To achieve this, they
must provide evidence that the system is
designed, constructed, and tested according
to best practices implemented throughout the
various phases of the lifecycle.

The Benefit:
Faster, Easier, Better Documented Compliance
 Time saved doing tedious tasks
 Better consistency
 Easy traceability and auditability
 Precisely fit to their needs
”With Parasoft’s solution, previously
arduous, boring and difficult-to-document
tasks were transformed into tasks that we
could perform systematically—
in an automated and rapid manner.“

Customer Challenge: Streamline Certification for
Medical Device Software under IEC 62304
“Our products are used intraoperative. If any
failure occurs during an operation, the operation
might have to be aborted. Moreover, since we
monitor nerves and their signals, incorrect
interpretations and decisions made by our
software could lead to wrong decisions by the
user...and could cause injuries for the patient.”
The US FDA accepts IEC 62304 compliance as
evidence that medical device software has been
designed to an acceptable standard.

The Solution: Automating Risk Management
Processes with Parasoft
 Multiple languages: C, C++, .NET
 Integration with existing systems: IDE, bug-
tracking, source control

The Benefit: Increased Efficiency through
Automation
“The current level of verification for each
requirement task (including task pass/fail
status and coverage) is bi-directionally
traceable to all associated tests. Full
traceability is crucial for IEC compliance.
Using the Parasoft solution significantly
increases our efficiency because many
manual steps could be automated.“

Parasoft Support for FDA
Static analysis
Policy definition management
Requirement definition and tracking
Unit test
Peer review
Runtime error detection
Coverage

Summary
Financial and health risks are increasing
FDA requires a rigorous mature process
Static analysis both provides tight controls and prevents
real problems from occuring
Proper configuration and deployment reduce common
static analysis problems

FDA Resources
 FDA - “General Principles of Software Validation”
http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/Guidan
ceDocuments/ucm085281.htm
…guidance outlines general validation principles that the Food and Drug
Administration (FDA) considers to be applicable to the validation of medical
device software…
 FDA – 21 CFR 820
 http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch
.cfm?CFRPart=820&showFR=1
…Quality System Regulations
 ANSI/AAMI 62304
“Medical device software-Software life cycle processes”
http://webstore.ansi.org/
…A recommended practice provides guidelines for the use, care, and/or
processing of a medical device or system.

 Blog: http://alm.parasoft.com
 Web: http://www.parasoft.com/jsp/resources
 Facebook: https://www.facebook.com/parasoftcorporation
 Twitter: @Parasoft @CodeCurmudgeon
 LinkedIn: http://www.linkedin.com/company/parasoft
 Google+ Community: Static Analysis for Fun and Profit
IoT API's session - API World - San Jose, CA Sep
13th
Managing Auto Supply Chain Risk – Automotive
Software Kongress - Germany Sep 21-22
StarWest Conference - Anaheim CA - Oct 5-6

Rx for FDA Software Compliance

More Related Content

What's hot

Similar to Rx for FDA Software Compliance

More from Parasoft

Recently uploaded

Rx for FDA Software Compliance