Medical Device Cyber Testing to Meet FDA Requirements

1
Medical Device
Cyber-Testing
To Meet FDA Requirements
November 14 | 1 pm EDT

About Us – Complementary Partners
2
HackerOne is the global leader in human-powered, AI-enabled security, fueled
by the creativity of the world’s largest community of security researchers. Our
platform combines the expertise of our elite community and the most up-to-
date vulnerability database to pinpoint critical security flaws. Our integrated
solutions—including vulnerability disclosure programs, bug bounty,
pentesting, code security audits—ensure continuous vulnerability discovery
and management throughout the SDLC.
BG Networks equips embedded engineers and penetration testers with
easy-to-use software automation tools to streamline cybersecurity tasks
including hardening, detection, and testing. BG Networks automation
tools are designed to help with adherence to regulations from the FDA,
NIST, ISO, and the EU.
ICS supports our customers with software development, User experience
design, platform and regulatory support to build next generation
products. We provide a number of services focused on the medtech
space including human factors engineering with a 62366 compliant
process, hazard and risk analysis, 62304 compliant software
development, and platform support including cybersecurity.
Cybersecurity
Services
Cyber-Testing
Detection
Human-
powered
Security
Testing

Speaker Introductions
3
Jarret Raim
VP of Product
Colin Duggan
Founder & CEO
Milton Yarberry
Director of Medical Programs &
Cybersecurity

Cybersecurity in Medical Devices: Practical Advice for FDA’s 510(k)
Requirements Webinar Series
4
On Demand Practical Advice for FDA’s 510(k) Requirements
https://www.ics.com/webinar-demand-practical-advice-fdas-510k-requirements
On Demand A Deep Dive into Secure Product Development Frameworks (SPDF)
https://resources.ics.com/webinar/secure-product-development-frameworks
On Demand Secure-by-Design - Using Hardware and Software Protection for FDA Compliance
https://resources.ics.com/webinar/secure-by-design-hardware-software-protection
On Demand - Threat modeling and risk assessment – First step in risk management
https://resources.ics.com/webinar/threat-modeling-risk-assessment
Will Ask Poll Question At End About Which Topics You Would Prefer Next

Agenda
• Where does it say the FDA and MDR require cyber-testing?
• Cyber-testing & Secure Product Development Framework (SPDF)
• FDA-required cybersecurity testing types
• Examples of tests across platforms: IoT, Phone App, Cloud, SaMD
• Continuous vulnerability discovery throughout the product lifecycle
• An effective pentesting methodology
• Sign-up : 1 on 1, testing checklist session
5

Working Sessions – Sign Up During This Webinar
Cyber-testing check list review - prepare for testing
Offering educational/working sessions based on your questions & device
• We’ll review a check list of preparations steps for cyber-testing
• It will save you time getting ready for a FDA submission or MDR assessment
• After the session, we’ll give you the check list
Sign up on Calendly, at the link below, for a 30-minute session
• Here is the link and we’ll put it in the chat

Questions For Us And A Question For You
Questions for us:
• Put your questions in the Q&A
• For questions we don’t get to, we’ll write answers and make them available after
A question for you:
What aspects of FDA and MDR cyber-testing, including pentesting, would you like to learn more about?
7
POLL QUESTION RESPONSES (please respond now - may choose more than one)
a. Efficient working with external testing teams
b. Collaborating with an internal testing team
c. IoT device cyber-testing
d. Smart phone application cyber-testing
e. Cloud cyber-testing
f. SaMD cyber-testing
g. Difference between pentesting and bug bounties

Cybersecurity Testing Is Required for Medical Devices
Section 524B of the FD&C Act (AKA The Patch Act)
FDA Guidance Cybersecurity in Medical Devices: Quality System
Considerations and Content of Premarket Submissions
• “Security testing documentation and any associated reports or
assessments should be submitted in the premarket submission.”
EU: Medical Device Regulation (MDR)
MDCG 2019-16 : Guidance on Cybersecurity for medical devices
• “The primary means of security verification and validation is
testing. Methods can include security feature testing, fuzz
testing, vulnerability scanning and penetration testing.”
• Also tool for “secure code analysis”, scanning for open source
and identify components with known issues

As an Aside - Commercial Testing Requirements
9
EU Cybersecurity Resiliency Act (CRA) – adopted in October
US Executive Order: 14028 Improving the Nation's Cybersecurity
SECURITY REQUIREMENTS RELATING TO THE PROPERTIES OF PRODUCTS WITH DIGITAL ELEMENTS
1) Products with digital elements shall be designed, developed and produced in such a way that they ensure an appropriate
level of cybersecurity based on the risks;
2) Products with digital elements shall be delivered without any known exploitable vulnerabilities;
3) On the basis of the risk assessment referred to in Article 10(2) and where applicable, products with digital elements shall:
a) be delivered with a secure by default configuration, including the possibility to reset the product to its original state;
b) ensure protection from unauthorised access by appropriate control mechanisms, including but not limited to authentication, identity or access
management systems;
c) protect the confidentiality of stored, transmitted or otherwise processed data, personal or other, such as by encrypting relevant data at rest or in transit
by state of the art mechanisms;
d) protect the integrity of stored, transmitted or otherwise processed data, personal or other, commands, programs and configuration against any
manipulation or modification not authorised by the user, as well as report on corruptions;
e) process only data, personal or other, that are adequate, relevant and limited to what is necessary in relation to the intended use of the product
(‘minimisation of data’);
>> Apply “effective and regular tests”
• No specific testing requirements
• Direction setting
• Will require verification once implemented
…

Testing in the Secure Product Development Framework (SPDF)
10
M
Design Controls
Design Inputs
Cyber ReqA
Cyber ReqB
Design outputs
Cyber SpecX
Cyber SpecY
Cyber SpecZ
Binaries
Verification Tests
Cyber TestX
Cyber TestY
Cyber TestZ
Mitigations
MitigationX
MitigationY
MitigationZ
Threat Assessment
ThreatX
ThreatY
ThreatZ
Security
Architecture
Architecture Diagrams
Component Analysis
Connectivity definitions
Use Case Views
Code
Known
Abnormalities
(test failures)
Static
Software
Code
Analysis
Source
SCA
Binary
SCA
SBOM
Triage &
Justifications
Vulnerability
Report
Penetration Testing
(independent white hat)
Post Market
Vulnerability
Management Plan
Customer
Transparency Plan
Published
Vulnerabilities
Threat Mitigation
Testing
(vs. ReqA, ReqB)
Vulnerability
Testing
(i.e. malformed input, fuzzing, etc.)
Cybersecurity
Assessment
Security Risk
Management
Report
(PMA - Annual)
Security Risk
Management Plan
Security Risk Test
Plan
SPDF
composition
Mitigations
1
2
3
4

SPDF Listed in FDA’s Guidance
All Are Based On the Same Testing Principals – All Mention Fuzzing & Scanning
Medical Device and
Health IT Joint
Security Plan (JSP)
ANSI/ISA 62443-4-1
IEC 81001-5-1 Health
Software and IT
SECURITY RISK MANAGEMENT-
THREAT MODELING-RISK
ASSESMENT
Does not specify detailed, device-
specific risk assessment
procedures.
Risk assessment is covered by
62443-3-2 at system level.
Industrial Automation Focused.
Healthcare regulations aligned.
Specific guidelines for assessing
the risks.
Comprehensive and healthcare
software specific. Borrows from
62443-4-1. Not device specific.
Includes explicit steps.
SECURITY ARCHITECTURE –
IMPLEMENTATION OF RISK
CONTROLS
Focusing on embedding security
features during the development
phase and ensuring they are
effectively implemented and
maintained.
Explicit requirements for
assessment, implementation and
continuous management of
security controls.
Comprehensive requirements for
controls to protect patient data,
system integrity and availability.
CYBERSECURITY TESTING Requires product validation and
verification security testing,
including development and
maintenance phases
Requirement, Threat,
Vulnerability, Penetration
Testing is heavily emphasized for
all development lifecycle
Including Vulnerability testing,
Fuzz testing
Pentesting on the product.
Robustness Testing
Clear requirements for continuous
comprehensive testing maintaining
the integrity and security, including
Fuzz and Pentesting
SPDFs

FDA eSTAR
Requires Submission of a Testing Document
The help (accessed via the “?”) text here is an exact duplicate of most of the content in the FDA’s Cybersecurity in Medical Device:
Quality System Consideration and Content of Premarket Submissions in section V.C.
(Security Requirement, Threat Mitigation, Vulnerability Testing, Penetration Testing)

13
What the FDA & Says About Testing
Classic Verification & Validation But Also Specify Security Testing Required
A closer look at the rationale’s for testing
SECURITY REQUIREMENTS (verify design)
CFR 820.30(f), a manufacturer must establish and
maintain procedures for verifying the device design.
THREAT MITIGATION (validate design)
21 CFR 820.30(g), a manufacturer must establish and
maintain procedures for validating its device design.
VULNERABILITY TESTING
Testing against know vulnerabilities. Techniques
often used include fuzzing, scanning, robustness
across the attack surface
PENETRATION TESTING
Performed by independent testers (i.e., not involved in
the design) using approaches that adversaries (i.e.,
hackers) would use.
FDA’s Four Types of Cybersecurity Testing
This is a Regulation
Threats → Controls → Verify effective controls (did it work?)
Tracing documentation
This is a Regulation
Security Requirements implemented successfully
Tracing documentation
** Boundary Analysis and rationale of boundary assumptions
Known vulnerabilities – with high evidence expectations
Abuse case, Attack surface analysis, Vulnerability chaining, Closed box
testing, Software composition analysis, Static and dynamic analysis
Independent Attacker perspective
Diversity is a plus
Could have some overlap with Vulnerability Testing

Why Is Cyber-Testing Different Than Standard Software Testing
Standard software testing
• Features/requirements performing as intended
• Corner-case testing with no unintended ‘features’ bugs
• Testing at different levels with different methods (unit, integration)
• Static/dynamic code analysis
Cybersecurity testing
• Includes standard testing for cyber features/requirements
• But, also includes
• Methods used by attackers
• Known vulnerabilities and exploits
• And,
• Independence is important – how hard are you trying to break it?
• Cybersecurity is deeply specialized – many layered domains
• Qualifications are exceptionally important – many different domains
Safety vs. Security, A different type
of threat
• Medical device → Safe and
Effective
• So if it isn’t safety related,…is it
important?
• FDA Guidance: ”Cybersecurity is Part of
Device Safety and the Quality System
Regulation”
• Security addresses a different set of
harms
• … harm, through loss of data, uncontrolled
access to data, corruption or loss of
diagnostic information , or corruption of
software leading to malfunction of the
device
• Cyber testing prevents your product
being an attack surface into a larger
system

Fuzzing vs Scanning : What is the Difference
Scanning is looking to acquire information that you use for some purpose
• You are looking for something specific
• Example: What ports are open and you get a list (e.g., NMAP)
• There is something that you are looking for that you already know what it is
• May check for all of something (testing for all ports open)
• Discovery process
• You know there is a common vulnerability, you do x,y,z, and it causes something to
respond incorrectly, you scan/check for that vulnerability
Fuzzing – trying to do something outside the normal area and cause the device to do something
that it should not
• Test to see if the device crashes, or runs slowly
• Fuzzing is testing for things that you don’t know is there
• Bombarding an interface with data to see if it is failing
• Try to cause something to crash
• You don’t know what you are looking for

Threats to Consider
Will Use Some Of These In Examples
Threat % Mentioned in Poll*
Extraction of patient information 54%
Denial of service 51%
Manipulation of software 35%
Compromise of Passwords 30%
Ransomware 27%
Supply Chain Attack 27%
Physical Tampering 24%
Disruption Cloud Operation 24%
Compromise of Software Update 22%
Manipulation of system(configuration) parameters 22%
* Secure-by-Design Webinar Poll Question: What cyber-threats, the drivers of
risk, are you most concerned about, and do you see your devices facing?
Top 10 Threats Mentioned In Poll from June
Medical Device Cybersecurity Webinar

Traceability
Security Requirements and Threat Mitigation Testing – One For One to Requirements & Threats
Threat Example Attack Path Security Requirement
Extraction of patient information Cloud S3 bucket open Remove/block Public Access in all S3
permissions. Permissions only changed
by authorized personnel.
Denial of service Communication flood via Bluetooth the device to drain
battery of IoT Medical Device
Only connect to authorized and
authenticated devices and limit scanning
interval
Manipulation of software Man in the Middle attack where a Smartphone App is
updating an IoT Medical Device
Update to IoT device need to be checked
for authenticity and integrity
Compromise of Passwords Unauthorized person gains access to a PC and SaMD
software by taking advantage of weak passwords
Multifactor authentication to use SaMD
software.
Security Requirement & Threat Mitigation (a.k.a V&V) need to be tied back to Security
Requirements & Threat Model

1) Security Requirements Testing (Verification Testing)
Testing that shows each design input requirement was implemented successfully
Threat Example Attack Path Security Requirement Security Requirement Tests
permissions. Permissions only changed by
authorized personnel.
Test all AWS Public Access Settings to be set to
block public access.
Test all AWS Public Access Permissions have
been removed and ACLs don’t allow public
access.
Denial of service Communication flood via
Bluetooth the device to drain
battery of IoT Medical Device
authenticated devices and limit scanning
interval
Test that IoT Medical Device authenticates before
starting communications and for scanning interval
configuration
Manipulation of software Man in the Middle attack where
a Smartphone App is updating
an IoT Medical Device
Software update to IoT device need to be
checked for authenticity and integrity
Test that software update is signed.
Test that software updates is checked for
authenticity and integrity before installation.
Compromise of Passwords Unauthorized person gains
access to a PC and SaMD
software by taking advantage of
weak passwords
Multifactor authentication required before
running SaMD
Test that SaMD software on startup requires MFA
on all operating systems supported

2) Threat Mitigation Testing (Validation Testing)
Testing that provides evidence that risk controls are effective
Threat Example Attack Path Security Requirement Threat Mitigation Testing
permissions. Permissions only
changed by authorized personnel.
Test externally to see if access can be gained
access.
Test within the AWS environment S3
permissions can be change
Denial of service Communication flood via Bluetooth the
device to drain battery of IoT Medical
Device
authenticated devices
Test for basic authentication bypass.
Test authentication without correct signatures.
Test beaconing in shorter intervals than
scanning is configured for
Manipulation of software Man in the Middle attack where a
Smartphone App is updating an IoT
Medical Device
Software update to IoT device need to
be checked for authenticity and
integrity
Attempt to install software that is not correctly
signed.
Compromise of Passwords Unauthorized person gains access to a
PC and SaMD software running on it by
credential stuffing and is able to steal
patient data.
Multifactor authentication required
before running SaMD
Test that MFA is required for all modes/state
software might be in

3) Vulnerability Testing
A list of testing methodologies in the FDA’s September 2023 guidance,
that includes testing for known vulnerabilities.
The specific testing methodologies that the FDA lists:
• Robustness
• Fuzz testing
• Attach surface analysis
• Vulnerability chaining
• Close box testing of known vulnerability scanning
• Software composition analysis of binary executable files
• Static and dynamic code analysis

4) Penetration Testing
A tester, who is independent from the design, takes on a mindset an
adversary would have, to find security weaknesses before attackers do.
FDA asks for the following pentesting information be documented and submitted:
• Technical independence
• Expertise of testers
• Scope of testing
• Duration of testing
• Test methods employed
• Test results/findings/observations
A few of my favorites illustrating pentesting techniques – a paper & two videos
• https://spqrlab1.github.io/papers/icd-study.pdf – Pacemaker hack
• https://www.youtube.com/watch?v=OobLb1McxnI – Classic testing techniques/exploits for IoT
• https://www.youtube.com/watch?v=96mpTFWWO7I – Buffer overflow exploit

White vs Gray vs Black Box Testing
Information Test Teams Need From You
We prefer White Box or Gray Box but Black Box testing maybe appropriate
• White box - All/most information about the system is provided including source code
• Gray box – Partial system information is provided
• Black box – No information is provided
With information about the system, we can be more efficient finding
vulnerabilities as less reverse engineering is needed

Save Expense & Improve Security: Advanced Testing Techniques
Shifting Left – Remote Testing – Automation – Crowdsourced Testing - Bug Bounties
Phase: Development
Shift Left! Vulnerabilities found earlier
cost less to fix
Phase: Prep for Premarket Submission
Enhanced through remote access and
automation
Enables:
• Secure testing of devices from anywhere
• Around the clock testing
• Automated re-testing to confirm if
vulnerabilities fixed
Phase: Pre & Post Market
Crowdsourced testing, bug-
bounties: Leverage 1000s of researchers
Enables:
• Diversity of expertise, methods, & tactics
• Bug bounty gives continuous testing over
extended periods
• Finds most difficult vulnerabilities
Pentesting
Faster time to completion
Shift Left!
Initial security testing within CI/CD
Software
With potential vulnerabilities
Identified Vulnerabilities
Crowdsourced Testing & Bug Bounty
Continuous discovery: Pre-market to post-market
Application SW
Operating System (OS)
Boot Loader
Rootfs
Device Firmware

We're trusted with the
businesses of hundreds of
thousands of merchants all
over the world. We want to
make sure that we're doing
everything we can for the
security of our platform.” -
Shopify
There are a lot of cases of
how we build a platform
and how we expect the
end users to interact with
them. This is very different
from how the research
community looks at it
and they found a lot of
unexpected outcomes as
a result.” - Epic Games
Why Leading Brands Choose HackerOne
“Hackers help us drive
digital transformation
and innovation, by
bringing a unique
outsiders perspective
to how we build and
run our infrastructure.
“By having an
open and
transparent bug
bounty program,
we ensure that if
someone finds a
security issue,
they come to us
with it first.
The findings from the
program help enhance our
preventative security efforts
from the inside out. Our
engineering team reviews
each report, prioritizes
according to the severity,
and uses the data to better
understand and protect
against malicious hackers.” -
Salesforce
“From an ROI
perspective, bug
bounty is one of
the most effective
programs in our
security strategy.
Trust
Innovation
Results
24

25
25
Continuous Vulnerability Discovery

26
HackerOne Code Security Audit
HackerOne Pentest
HackerOne Challenge
HackerOne AI Red Teaming
26
HackerOne Bounty
HackerOne Response

27
HackerOne Code Security Audit
HackerOne Pentest
HackerOne Challenge
HackerOne AI Red Teaming
27
HackerOne Bounty
HackerOne Response

28
Leverage the diverse
expertise of the largest
community of security
researchers and
pentesters to uncover
elusive vulnerabilities
that only human
creativity can detect.
World’s Largest
Ethical Hacking
Community
Valid vulnerabilities
submitted
in the last 12 months
1 in 5 Vulnerabilities found
with a high/critical
rating
1 Critical vulnerability
found every hour
28
80K

1
Pentester Selection and Feedback Loops
Onboarding & Skill-Matching
Approved pentesters are onboarded and get assigned to best-suited engagements based
on their skills and experience.
Vetting & Verification
All pentesters are HackerOne Clear verified and undergo a rigorous criminal background screening.
Selection
HackerOne ranks applicants based on criteria specified in the Pentest Community Application.
Continuous feedback loops ensure testers deliver consistent, high-quality results
Testers join dedicated Slack instances to collaborate with TEMs and customers
3
2

30
HackerOne Pentesters
Our pentesters are a cohort made up of
the most elite security researchers
globally. They all undergo an advanced
vetting process, possess diverse technical
expertise, and in-depth knowledge of
compliance frameworks.
Pentesting and Industry Experience
Leandro
@none_of_the_above
Leonel
@delisyd
Miguel Regala
@fisher
Joel
@niemand_sec
Trev
@SoWhatSec
Rodrigo
@rororodrigo
Protect Critical Assets with Expert Pentester Skills

HackerOne Pentest
31
HackerOne Pentest is a methodology-driven
security testing solution delivered via a Pentest
as a Service (PTaaS) model.
■ Quick launch and easy scoping
■ Elite pentesters, skill-matched to your needs
■ Real-time collaboration and SDLC
integrations
■ Superior Zero Trust Network Access (ZTNA)
and testing control
■ Accelerated remediation
■ Easy retesting and repeat engagements

HackerOne’s Pentesting Methodology
32

Pentesting for Web and Mobile Apps
Web App Testing
▪ Test for broken access control,
cryptographic failures, and
security misconfigurations
using the OWASP Top 10.
▪ Ensure secure handling of
patient data and compliance
with HIPAA and HITRUST by
leveraging specialized
healthcare testers.
▪ Focus on protecting sensitive
health records from
unauthorized access and
breaches.
iOS and Android Testing
▪ Test healthcare apps,
focusing on PHI protection
and compliance with
healthcare standards.
▪ Leverage OWASP Mobile
Top 10 to address injection
flaws, insecure data
storage, and
communication.
▪ Ensure strong encryption
and secure data handling,
adhering to HIPAA and
HITRUST requirements.

Pentesting for Cloud
AWS Testing
▪ Focus on identifying
vulnerabilities in AWS services
hosting healthcare data,
including API Gateway, S3, and
IAM Roles.
▪ Ensure secure API
management, encrypted data
storage, and tight access
controls to safeguard PHI.
▪ Engage testers with expertise in
healthcare cloud environments
to ensure alignment with HIPAA
and HITRUST standards.
AWS Security Config
Review
▪ Review AWS security
settings to align with
healthcare best practices
and HIPAA/HITRUST
compliance.
▪ Examine services like IAM,
S3, and CloudTrail for
secure access
management, logging,
and data protection.
Azure Security Config
Review
▪ Verify Azure security
settings meet HIPAA and
HITRUST requirements for
protecting patient data.
▪ Focus on Entra ID, Azure
RBAC, and Blob Storage to
ensure secure data
storage, access control,
and audit trails.

35
35
Digital Healthcare Security Success with HackerOne
Security Goals
● Embed a security-first culture by
creating champions across
departments to prioritize patient
data protection and compliance
● Identify vulnerabilities
continuously through regular
pentesting and bug bounty
programs to safeguard sensitive
health information
● Optimize spend on high-impact,
actionable results
Why HackerOne
● Leverage a global community of
ethical hackers and pentesters to
uncover critical healthcare-specific
vulnerabilities, ensuring
comprehensive protection of
sensitive patient data.
● Streamline remediation with
integration into workflows and
compliance with HIPAA and HITRUST.
● Enhance security posture through
continuous testing, combining
human intelligence with AI-driven
insights to preempt evolving threats
and maintain patient trust.
HackerOne Bounty | HackerOne Pentest | HackerOne SAS | HackerOne Triage
“An offensive security
testing program is the
highest ROI program you
can have. You're getting
hammered by the best
researchers. I'm really
impressed by the
skillsets of researchers
across the board.”
Security & Compliance
Director of a leading digital
healthcare provider

One More Poll Question – Which Topics Come Next
We’ve covered the following topics in webinars so far:
1) Practical advice for FDA cybersecurity
2) SPDF
3) Security by Design
4) Threat & Risk Assessments
5) Cyber-testing
Next Webinar is Planned for January
What topics would you like to see covered in future webinars?
36
POLL QUESTION RESPONSES (may choose more than one)
a. Defense in Depth – 8 Security Control Categories Required by the FDA
b. Artificial Intelligence – Cybersecurity Considerations
c. FDA vs MDR Cybersecurity – Both at Once?
d. Post Market Cybersecurity – What Needs to be Done ?
e. Software updates – Best Practice
f. Bolting on Security – Protecting Legacy Devices
If you have topic that is not listed,
please put it in the chat or let us
know via email

Questions?
38
Jarret Raim
VP of Product
Colin Duggan
Founder & CEO
Milton Yarberry
Director of Medical Programs &
Cybersecurity
jfaherty@ics.com

Medical Device Cyber Testing to Meet FDA Requirements

In this document

More Related Content

What's hot

Similar to Medical Device Cyber Testing to Meet FDA Requirements

More from ICS

Recently uploaded

Medical Device Cyber Testing to Meet FDA Requirements