Disaster Recovery Planning

DISASTER RECOVERY PLANNING FOR HEALTHCARE
Ron Pelletier, CISSP, CBCP, CISA, QSA
VP and Executive Manager
FOR HEALTHCARE ORGANIZATIONS
DISASTER RECOVERY PLANNING
INMGA – August 11, 2015

AGENDA
• Disaster Recovery Planning Overview
• General DRP Considerations
• A BCM Methodology (encompasses DRP)
• Trends and Standards
• Questions
PONDURANCE 2

DISASTER RECOVERY PLANNING FOR HEALTHCAREPONDURANCE 3
DISASTER RECOVERY PLANNING OVERVIEW

SIMPLIFIED DRP TERMS
PONDURANCE 4
Disaster Recovery – Planning to sustain supporting technology & data.
Crisis Management – Preserving life safety and business image.
Business Impact Analysis – Establish the organization’s critical path.
Recovery Time Objective – When do the systems/processes need to be restored?
Recovery Point Objective – How much data can you stand to lose?
Maximum Tolerable Downtime – What is the point of unacceptable risk?
Risk Tolerance – Collective picture of risk management and BCM.
High Availability – When downtime of systems/data is not an option.
Minimum Operating Requirements – What do you need, and when, to get by.

WHERE DOES DRP FIT IN THE BCM LIFECYCLE?
PONDURANCE 5
BCM
Business
Continuity
Planning
Disaster
Recovery
Planning
High
Availability
Risk
Management
Incident
Response
Crisis
Management
(general, not all inclusive)

TRADITIONAL THINKING ON DISASTER RECOVERY
PONDURANCE 6
Disaster Recovery vs. Business Continuity
PEOPLE
BUSINESS
PROCESSES
PROCESS
CONTINUITY
BUSINESS
PROCESSES
DRPDRPDRP
Disaster
Recovery
Business
Continuity
TECH/DATA
RESTORE
B
U
S
I
N
E
S
S
C
O
N
T
I
N
U
I
T
Y
B
U
S
I
E
N
S
S
C
O
N
T
I
N
U
I
T
Y

THE INTEGRATED PERSPECTIVE
PONDURANCE 7
Defined Tolerance for Risk
Program Exercising, Change Management, Maintenance
(BCP)
Business
Continuity
Planning
(DRP)
Disaster
Recovery
Planning
DRP
Strategies
BCP
Strategies
DRP
Documentation
BCP
Documentation
The Risk Analysis Phase
Current
State
Assessment
Threat and
Risk
Assessment
Business
Impact
Analysis
CRISIS MANAGEMENT
• Owns Initial and Ongoing Response
• Allocates Emergency Resources
• MAKES DECISIONS AS REQUIRED
• Functions as Steering Committee

GENERAL DRP CONSIDERATIONS

EVALUATE YOUR OPERATING RISK
PONDURANCE 9
• PREPARE TO AVOID BUT PLAN TO RESPOND!!
• Human, technical, operational and strategic threats MUST be considered to
formulate a viable avoidance and/or response posture
• Look for single points of failure that might not have been considered
(control systems, joined power junctions, shared data closets, shared
passwords, single communication gateway)
• Consider your level of reliance on other entities (parent organizations,
shared services, external service providers, etc.)
• Integrate your risk assessment process with Cyber Security efforts. According
to KPMG’s BCM Survey Only 41% of Companies integrate BCM with Cyber Security
• Do you have specific technologies at your site that are not typically
supported by a shared services organization?
• Do you have a defined owner or custodian for your Disaster Recovery
Planning efforts?

DEFINING RISK TOLERANCE FOR DRP
PONDURANCE 10
$ and Operational Impacts
Manual Processing
Application ‘X’ in 72 Hours Application ‘X’ in24 Hours
Management Negotiation
Based on Risk Tolerance
Recovery Time Objectives (RTO’s)
Recovery Point Objectives (RPOs)
Current Recovery Capabilities
(CRC’s)
Information Technology Group
Current State Assessment
Maximum Tolerable Downtimes
(MTD’s)
Business Unit Personnel
Business Impact Analysis

RISK TOLERANCE AND HEALTHCARE
PONDURANCE 11
• Two schools of thought can muddy the water when
considering technology downtime in healthcare:
o “We have been treating patients for centuries without
technology…we can live without it indefinitely”
o “We have grown so dependent on technology that we
cannot be inconvenienced by its loss for even a single
hour”
• How do you appropriately consider the risk to your
organization, without trying to “over-engineer” a solution?
• What happens if technology platforms are down for extended
periods of time?

RISK TOLERANCE AND HEALTHCARE
PONDURANCE 12
• Consider looking at 2 key criteria to arrive at true business impact:
1. Degradation of Care (Life Safety): The degradation of care considers
specific risks of patient-safety, if care professionals do not have
access to all patient records that may provide insight into patient
profiles (e.g., pharmaceuticals, allergies, past procedures, etc).
2. Patient Throughput (Financial and Operational): “Throughput”
represents the number of patients that can be reasonably and safely
treated over a given period of time. Without a level of automation
and record accessibility, it’s logical to assume that hospitals will not
be able to attend to, admit, or discharge the “normal” volume of
patients with the same level of efficiency. This can lead to direct
financial impacts, as it would likely lead to a reduced and untimely
level of billing for patient care.
• Use qualitative measures where they make sense, but attempt to arrive
at a Recovery Time Objective for each key system (HIPAA denotes this as
“addressable”)

DEVELOPING DRP ROLES/RESPONSIBILITIES
PONDURANCE 13
• Consider a 360 degree approach to ensure appropriate organizational
coverage
• Look outside the organization to determine if there are groups/entities
with whom you need to coordinate your strategies and plans
• If you are part of a hospital system, have you integrated with their
Hospital Command?
• If you have personal that you contract to facilities, do you know what
their plans are if their facilities are impacted?
• Break down the roles and corresponding plans to facilitate action and
accountability
• How do you define an incident commander?
• What about facilities? Specific technologies?

DEVELOPING DRP ROLES/RESPONSIBILITIES
(SAMPLE ORG CHART FOR DRP)
PONDURANCE 14
IT Incident
Commander
TBD at Time of Incident
IT Customer Support
Center Lead
IT Hospital Command
Liaison
IT
Safety/Security/Privacy
Officer
IT Command Group IT Operations Support Group
Infrastructure Team Leader
Applications Team Leader
Facilities Director
Logistics & Vendor Support
Finance/Administration
Hospital Command
IT Facility and Technical Teams
IT Facility
Coordinators
Applications
Teams
Infrastructure
Teams
Data Recovery
Teams
IT Security
IT Executive(s)

DRP SCENARIO PLANNING
PONDURANCE 15
• Ensure your response procedures have adequate flexibility to respond to
both common and unique situations
• Do not get too specific or box your plans to a certain scenario, use
situations that may prompt a certain response
• Align your response planning with the applicable Hospital Command (if that
is applicable)
• Remember that disasters related to technology could take on physical form,
logical form, or a combination of the 2
• While area-wide disasters are less likely to occur, they need to be at least
considered (think Hurricanes Sandy, Katrina; Northeast power outage; ice
storms, etc.)

DRP SCENARIO PLANNING - EXAMPLE
PONDURANCE 16

REVIEW YOUR DATA BACKUP AND RECOVERY
PONDURANCE 17
• Ensure the data backup scheme complements the Recovery Time and
Recovery Point Objectives (RTOs & RPOs)
• Tapes are fine, but often they are either not removed from the site or are
taken offsite 1x per week
• If the backups (tape or disk) are not tested periodically to verify full
restoration, the capability to restore is questionable
• If the backup tapes are not encrypted when removed offsite, you are
introducing a whole new set of risk
• Don’t blindly jump to a high availability strategy if it is not justified. It is
entirely possible that even a replication strategy is not necessary, and a
high availability strategy may completely over-engineer the program
• BUT…only proper analysis can provide that answer

DRP DOCUMENTATION CONSIDERATIONS
PONDURANCE 18
• Consider segmented action plan documents that are managed by
accountable person(s), but provide seamless integration and consistency in
format
• Have a Central Plan to drive communications and Emergency Response
• Have “extracts” or job action sheets that represent specific technical
procedures for rebuild, restore, recover, etc.
• Assign accountability as appropriate, and add depth to preserve continuity
• Ensure the procedures are fairly thorough, but do not drive inflexibility or
box the responders into a single set of actions
• Store the plans where they are accessible, particularly if your internal
systems fail
• Ensure the plan appendices have adequate reference information (key
vendors and contacts, location of stored equipment, etc.)

DRP DOCUMENTATION CONSIDERATIONS
PONDURANCE 19

A FULL BCM METHODOLOGY
(WITH DRP CONSIDERATIONS)

6 DOMAINS TO CONSIDER FOR BUILDING BCM
PONDURANCE 21
Assess the entity controls that integrate, manage, and sustain a viable BCM
throughout the enterprise
1. Program Management
•Program Definition – Establish the program is formally developed and integrated
•Support and Accountability – Establish the program is supported at the highest level of the org
•Budget Planning and Program Evaluation – The org is committed to sustaining program viability
The organization has defined its recovery, restoration, and high availability
requirements related to business processes, applications, infrastructure & data
2. Requirements
Definition
•Risk Analysis and Treatment – Establish the org has analyzed its risk posture, reasonably reduce risk
•The BIA Methodology – Establish the org maintains formal method to define impacts, prioritize criticality
•Data Flows and Dependencies – Establish that dependencies (internal/external) are documented
•Analysis and Reporting – Establish that BIA results are aggregated, prioritized, and approved
Assess the organization’s method for developing continuity and availability strategies,
within its maximum tolerable downtime.
3. Strategy Selection
•Staff and Support Requirements – Establish that strategies are developed based on defined requirements
•Course of Action Analysis – Establish that the cost to maintain strategies in line with risk tolerance
•Monitor, Evaluate for Change – Establish that strategies are periodically evaluated for change

6 DOMAINS TO CONSIDER FOR AUDIT
PONDURANCE 22
Assess the sufficiency, completeness, applicability, and implementation of the
organization’s documented BCP/DRP plans.4. Plan Development
•Plan Components & Framework – Establish plans are documented, align with requirements
•Supporting, Storing Plans – Establish plans are accessible, assigned to process owners
•Plan Updates – Establish plans change as processes, technologies, people change
Assess the organization’s method for vendor selection and oversight relevant to
the BCM program.5. Vendor Management
•Vendor Contracting – Establish vendors are screened, will meet contractual requirements
•Critical Vendor Dependencies – Establish critical dependencies are known, accounted for
•Vendor Integration, Testing – Establish vendors occasionally participate in tests/exercises
Assess the organization’s capability to test and maintain the viability of its BCM
program.
6. Implementation,
Maintenance
•Testing and Validation – Establish plans are valid through scheduled, ongoing testing
•Change Management – Establish changes required to BCM are formalized
•Workforce Awareness – Establish workforce members are aware of the BCM program

CONSIDER A MATURITY MODEL APPROACH
PONDURANCE 23
As of: SEPTEMBER 2012
Client:
Affiliate:
Maturity
Rating
Not Addressed
Minimally
Addressed
Emerging Managed
1 41% 0 0 5 7
1.1 25% 0 0 1 2
1.2 45% 0 0 2 3
1.3 54% 0 0 2 2
2 46% 0 2 10 4
2.1 25% 0 1 3 0
2.2 59% 0 0 0 4
2.3 25% 0 1 3 0
2.4 75% 0 0 4 0
3 61% 0 1 6 4
3.1 56% 0 0 3 3
3.2 47% 0 1 2 0
3.3 80% 0 0 1 1
4 38% 0 0 6 5
4.1 50% 0 0 4 2
4.2 40% 0 0 0 2
4.3 25% 0 0 2 1
5 30% 0 4 2 3
5.1 25% 0 0 1 2
5.2 40% 0 3 0 1
5.3 25% 0 1 1 0
6 67% 0 0 4 7
6.1 75% 0 0 1 3
6.2 50% 0 0 3 0
6.3 75% 0 0 0 4
47% 0 7 33 30
CLIENT NAME
SUB ORGANIZATION
QUANTIFIED BCM FINDINGS (# of findings per maturity level)
Vendor Contracting
Data Flows and Dependencies
Plan Updates
Supporting and Storing the Plans
Program Definition
REQUIREMENTS DEFINITION
The BIA Methodology
Support and Accountability
Budget Planning and Program Evaluation
Risk Analysis and Treatment
Analysis and Reporting
STRATEGY SELECTION
Change Management
Workforce Awareness
Enterprise BCM Principles
Critical Vendor Dependencies
Vendor Integration and Testing
PLAN IMPLEMENTATION & MAINTENANCE
Testing and Validation
Scoring
PROGRAM MANAGEMENT
Staff and Support Requirements
VENDOR MANAGEMENT
Course of Action Analysis
Monitor and Evaluate for Change
PLAN DEVELOPMENT
Plan Components and Framework
• Facilitates Scalable Program
• Isolates Highest Risk Areas
• Accounts for areas to sustain
• Incorporates All Findings from the Audit

DRP TRENDS & STANDARDS

EMERGING TRENDS IN DRP
PONDURANCE 25
• Virtualization helps reduce number of overall IT assets, improves
system uptime…but beware of single points of failure!
• Cloud computing provides a viable outsourcing option for production
technologies…but be sure your cloud vendor is capable of meeting your
RTOs, RPOs!
• Mobile devices provide a means of portability for documented plans,
communications, and rapid response…but be sure phones are secure,
encrypt if possible!
• Social networking provides an effective way to broadcast incidents,
particularly for crisis management…but be sure that the messages are
controlled!

CURRENT AND EMERGING STANDARDS
PONDURANCE 26
• Business Continuity Institute - Good Practice Guideline (2010)
• BS 25999 Business Continuity – BSI’s practices guideline
• Disaster Recovery Institute (DRI) – Professional Practices for BCM
• ISO/IEC 22301:2012 – One of the newest, aligned with the ISO27k set of
standards

Ron Pelletier, CISSP, CBCP, CISA, QSA
VP and Executive Manager
QUESTIONS
ron.pelletier@pondurance.com
www.pondurance.com
Pondurance
3105 East 98th Street
Suite 120
Indianapolis, IN 46280

Disaster Recovery Planning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (11)

Similar to Disaster Recovery Planning

Similar to Disaster Recovery Planning (20)

Recently uploaded

Recently uploaded (20)

Disaster Recovery Planning