What is a Disaster? Any Situation That Impedes On Day-to-Day Operations Natural Disaster • Tornadoes, severe winter storms, earthquakes, fires, dam failure, (floods and water leaks are statistically the number one threat), etc. Man-Made Disasters • Disgruntled employees, spouses, significant others • Union Strikes • Hazardous material spills • Terrorist (Foreign and Domestic) • Construction workers cutting power communication lines • Biological, chemical, nuclear devices • Civil uprisings Technical Disasters • Hackers, cyber-terrorism, power outages, voice and data communications line failure, software and hardware failures
What is Disaster Recovery?Business Continuity Program: An ongoing process supported bysenior management and funded to ensure that the necessary steps aretaken to identify the impact of potential losses, maintain viable recoverystrategies and recovery plans, and ensure continuity of servicesthrough personnel training, plan testing, and maintenance.Business Continuity Planning (BCP): Process of developing advancearrangements and procedures that enable an organization to respond toan event in such a manner that critical business functions continue withplanned levels of interruption or essential change. MANY SIMILARTERMS: Contingency Planning, Business Resumption, Resiliency.IT Disaster Recovery Planning (DRP): Process of developingadvance arrangements and procedures that enable an IT department torespond to an event in such a manner that critical business functionscontinue with planned levels of interruption or essential change.
Emergency Preparedness and ResponseWORKING WITH THE PUBLIC SECTOR WHILE RESPONDINGStrategic Phase: A plan set to identify who performs what function,when and how.-Establish Relationships - Police, Fire, Medical-Develop a monitoring and reporting process – Command and Control-Identify the first response teams-Review and receive signoff, establish Emergency Operations CenterTactical Phase: Strategy for dealing effectively with the emergency.-First Responders - (Incident Command System) - Evacuation - Medical care and personnel counseling - Hazardous material response - Fire fighting - Internal and external communication- Emergency Stabilization - Site safety, security, salvage, restoration- Facility Stabilization
Emergency ResponseTRANSIT IN EMERGENCY RESPONSE:Public transit agencies have a history of providing assistanceduring crisis situations, performing vital services such asevacuation of victims and transport of emergency personnel. Inthe aftermath of major disasters, public transit systems have oftensupplemented or replaced damaged or blocked roadways,maintaining mobility for residents and for repair and recoveryworkers.
Emergency Preparedness and ResponseDisaster Response and Recovery Resource for Transit AgenciesContents – From FEMA Homeland Security Office1. Introduction and Background How to find Information and Resources in this Document2. Frequently Asked Questions 2.1 Info for Transit Providers in Affected Areas 2.2 Info for Transit Providers serving Displaced/Relocated Persons 2.3 Charter Service Requirements 2.4 Emergency Transportation for Persons with Special Needs (including ADA Paratransit) 2.5 Funding Eligibility and Reimbursement 2.6 How to Help Emergency Evacuees 2.7 Assisting Special Needs Populations3. The Role of Federal Agencies and States in Disaster Response 3.1 The National Response Plan and NIMS. 3.2 State Emergency Management Plans 3.3 FEMA 3.4 FTA4. Local Disaster Response Resources and Best Practices 4.1 Introduction and Background 4.2 Emergency Preparedness: Planning and Best Practices 4.3 Disaster Response 4.4 Disaster Recovery 4.5 Characterizing Possible MPO Roles in System Operations and Security/Disaster Planning 4.6 Providing Emergency Services to Persons with Special Needs 4.7 The Transit Industry at LargeAppendix A: FTA Emergency Response Personnel Contact ListAppendix B: FEMA and State Contact List
Emergency Preparedness and ResponseNATIONAL TRANSPORTATION RECOVERY STRATEGY (NTRS)The National Transportation Recovery Strategy (NTRS) is designed to help transportation industrystakeholders and local, tribal, and State government officials prepare for and manage thetransportation recovery process following a major disaster.The overall goal of this Strategy is to promote a recovery process for transportation networks– and subsequently of communities in general – that results in a greater level of resilience.
Transit Helps in Emergencies HeadlinesAs Washington’s Columbia River and nearby waterways threatened to flood in February 1996,C-TRAN of Vancouver began monitoring water levels and planning with other local agencies foremergency services. When flood waters began to affect rural roads, C-TRAN detoured its routesto keep service running. As streets and bridges in Vancouver and Portland became hazardous,C-TRAN’s urban routes began early and increased commuter service to get residents home; forseveral days, mass transit was the primary mode of travel in downtown Portland. In addition,buses performed emergency evacuations and transported emergency and recovery personnelthroughout the crisis (1). During the following year, C-TRAN evacuated and sheltered Vancouverresidents during two chemical spills and a downtown fire (2).C Harrisburg, Pennsylvania’s Capitol Area Transit (CAT) responded to a variety of emergencyconditions during the blizzard of 1996 and its aftermath. From a sudden increase in transportationdemand when all government employees were sent home during the blizzard, to the evacuation ofresidents in flood zones, to the transport and shelter of firefighters during a four-alarm fire in lateJanuary, CAT vehicles and employees made significant contributions to Harrisburg’s winter stormresponse and recovery (3).C After the bombing of the Alfred P. Murrah Federal Building in Oklahoma City, Metro Transitbegan running 24-hour service to accommodate transportation needs. In addition to maintainingall regular service, Metro Transit buses transported firefighters, rescue teams, and medicalpersonnel, and evacuated residents from a nearby housing complex. Metro Transit personnel alsomanned the Multi-Agency Command Center, which coordinated communications during reliefefforts (4).C The 1989 San Francisco earthquake destroyed some of the area’s primary traffic arterials anddamaged others to the point of impassability. The San Francisco-Oakland Bay Bridge and the I-880 freeway, which together comprised the main connection between the cities of San Franciscoand Oakland, were closed after sections of these roadways collapsed. Several other freeways and
TAKING ACTIONErie PA Metropolitan Transit AuthorityErie Metropolitan Transit Authority (EMTA) has made significant investmentsover the past few years in its IT infrastructure to increase operationalknowledge, vehicle tracking, and overall efficiency of the organization.Furthermore, EMTA is a major resource during emergency events, such asman-made and natural disasters. EMTA has the vehicles, professional drivers,and systems to assist with large-scale evacuations and to provide shelter-in-place facilities as rest and recovery stations for first responders. In order forEMTA to ensure that these critical systems are available during a disaster andin order for EMTA to assist with emergency recovery and evacuation efforts,EMTA engaged DR vendor to develop an all-hazards Disaster Recovery Plan(DRP) to ensure that critical business functions continue during an emergency
PREPAIRINGPioneer Valley Transit AuthorityConduct Disaster DrillSPRINGFIELD, MA.,-The mock disaster scene will consist of an armored carand bus accident. The bus, with 22 passengers, is hit by the armored car whichtheoretically was used in a robbery, flips on its side, doors against thepavement, trapping the passengers inside.First responders participating in the drill are Springfield fire and police,American Medical Response, Mercy Medical Center, Pioneer Valley RedCross. The PVTA has worked closely with Springfield’s Director ofEmergency Preparedness, Robert Hassett to plan.“The goal is to build on, and maintain, good and open relationships with firstresponders in Springfield. It is important fire and police are familiar with ourbuses, and it’s important for us to test response time, communications, rescueand recovery.The Federal Transit Administration requires all Regional Transit Authorities todevote 1% of capital expenses toward safety and security.
PROTECTING DATAUtah Transit Authority Relies on Data Backup Vendor for DisasterRecoveryImproves its object storage archive to assure regulatory compliance andseamlessly support its disaster recovery strategy. Using disk based data backupUTA has dramatically reduced its backup window from five days to one whileimproving the overall performance, integrity and availability of its data, videoarchives, and CAD drawing archives.As with most public transportation agencies, a large volume of the data UTAmust protect comes from video surveillance of the stations and vehicles itoperates. Literally mountains of surveillance video can be captured every day.This combined with heavy operational usage of unstructured data was creatingvolumes of data that UTA needed to efficiently and effectively archive to meetcompliance and regulatory requirements.
FUNDINGCINCINNATIArticle from: US Fed News Service, Including US State News | August 24,2011 WASHINGTON, Aug. 23 -- The office of Sen. Sherrod Brown, D-Ohio, has issued the following news release:New safety improvements will be made to protect southwest Ohios publictransportation system from potential disasters and other emergencies. U.S. Sen. Sherrod Brown (D-OH) today announced that new federal resourceswere awarded to Southwest Ohio Regional Transit Authority (SORTA) createa disaster preparedness plan to protect Ohioans from acts of terrorism, majornatural disasters, and other emergencies."Our states public transportation systems are critical for connecting Ohioanswith schools, health care facilities, and employment opportunities,"
Being Ready to Apply for FundingSouth Jersey Transportation Authority applies for $1 million in disasteraid following Hurricane IreneThe South Jersey Transportation Authority is seeking federal disaster fundingto offset an estimated $1.1 million in lost revenue from Hurricane Irene,including about $320,000 from waiving toll collections on the Atlantic CityExpressway. The authority is applying to the Federal Emergency ManagementAgency to cover some of the losses. .The late August hurricane cost an estimated $905,000 in total toll revenues.State officials suspended tolls on the expressway and the Garden StateParkway to aid evacuations.Other hurricane costs included $130,000 in emergency staffing levels, as wellas $35,000 in losses, including lost landing fees at Atlantic City InternationalAirport, SJTA spokeswoman Sharon Gordon said.These included a three-day shutdown of Atlantic City casinos and a mandatoryevacuation of Cape May County and other shore towns at the height ofsummer.
Why Should You Develop A BusinessContinuity and Disaster Recovery Plan? As a Leader in your OrganizationPROTECT YOUR REPUTATION
Why Should You Develop A Business Continuity Program? Protect the Organization’s Assets • People, Equipment, Information (Data) Minimize damage and loss Minimize confusion, indecision Instills confidence in staff, public and customers Ensure employee welfare and safety Disaster Plan may be used for daily activities A Business Continuity Program saves TIME and MONEY responding to disasters Deal with the media in an appropriate fashion Expedite the return to “business as usual”
Business Continuity Methodology The Path To Successful Planning Recovery Analysis Interviews Project Planning Risk Assessments Observations Schedule and Kickoff Business Impact Analysis Recovery Strategy Options Data Collection fdsfdfs fdsfdfs Polic fdsfdfs ies and Proc Gui de edures Present Recovery Solutions Analyze Data Plan Consider Viable OptionsDevelopment Plan Testing Plan Enhancement Exercise Plan Maintenance Rehearsals
Business Continuity MethodsBackup and Restore of Information NO DATA NO RECOVERY
Business Continuity Methods Information Media Recovery Microfiche • Are they backed up and stored off-site? • Paper Records • Use fire proof filing or fire resistant filing cabinets • Use an imaging system Critical stand alone pc’s are they backed up? • Backup nightly - critical files to network storage, tape, or thumb drive/CD/DVDs *be careful while conducting incremental backups Is the IT department effective with data backups? Are backups tested? Offsite storage, NAS (network attached storage, SAN (storage area networks) VSN (virtual storage networks) Off-Site storage facility should be used for paper documents CDs, hard drives tapes, etc. (test your storage provider ask for a backup tape periodically) Fire proof vault for cash, checks, blank checks, contracts, insurance policies, etc.
RECOVERY ANALYSISCONDUCT A BUSINESS IMPACTANALYSISA management level analysis that identifies the impacts oflosing the entity’s resources. The analysis measures the effectof resource loss and escalating losses over time in order toprovide the entity with reliable data upon which to basedecisions concerning hazard mitigation, recovery strategies,and continuity planning.
RECOVERY ANALYSIS UNDERSTANDING Business Impact Analysis(BIA) Describes the business functions at the process level Identifies critical equipment (all the equipment you need to operate in disaster mode) Frequency of operations/functions • Continuously, annually, daily, weekly, etc. Identifies periods of high volume Financial, operational and service impacts identified Considers if job descriptions and operational procedures exist Sets business process priorities Identifies single-points-of-failure Do vendors have business continuity plans?
RECOVERY ANALYSIS UNDERSTANDING Business Impact Analysis(BIA)What are Critical Business Processes to Transit Authorities?Number ONE - PUBLIC SAFTEY• Fleet, Funding, Human Resources• Customer Services, Maintenance, Line Services• Fixed Routes, Scat Services, Special Services• Passport, Title, Speakers Bureau
RECOVERY ANALYSIS UNDERSTANDING Business Impact Analysis(BIA cont.)Recovery Time Objective (RTO) - The period of time that systems,applications, or functions must be recovered after an outage (e.g. one businessday). RTO’s are often used as the basis for the development of recoverystrategies, and as a determinant as to whether or not to implement the recoverystrategies during a disaster situation.CLASSIFY Priorities - Processes, Servers, Files Priority One, Two, Three, Four, Five Many organization use terms like Continuous Availability High Availability, Highly Recoverable, Less Critical to classify priorities business and computing priorities. Consider classifying new systems and operations as they evolve, turn BIA into part of the company lifecycle.Recovery Point Objective (RPO) - The maximum amount of data lossan organization can sustain during an event. Last backup till disaster.Recovery Time Actual (RTA) - The actual time it takes to recover abusiness function, consider gaps.
RECOVERY ANALYSIS QUESTIONWhat is the best way to recover from a Disaster?
RECOVERY ANALYSIS ANSWERNever have one in the first place! CONDUCT A RISK ASSESSMENT
RECOVERY ANALYSIS How to Prevent DisastersIdentify Hazards That May Cause A Disaster Mitigate The Identified Hazards
RECOVERY ANALYSIS CONDUCT A RISK ASSESSMENT Identifies vulnerabilities and ranks hazards/threats Examines all possible risk sources…physical security, systems security, facility, location, surrounding area The report will prioritize findings and recommendations for mitigation consideration Computer Based Security Assessment Tools are recommended starting points for computer security risk assessments
RECOVERY ANALYSIS CONDUCT A RISK ASSESSMENT Items To Assess Uninterrupted Power Supplies and Power Generators • In a secured location, • Is it tested regularly • Fuel contract (refill after testing) and a major supplier of fuel and an alternate Fire Suppression System Wet or dry pipes Fire extinguishers and usage training
RECOVERY ANALYSIS CONDUCT A RISK ASSESSMENT Items To Assess Physical facility security Electrical power grid feeds Telecommunication central offices used Multiple voice and data communication providers routing through same central office Evaluation of data center and network security vulnerabilities Virus protection, trojans, worms, adware/spyware detection, unnecessary open ports and services being used on servers, workstations and network equipment, identify opportunities hackers would use to attack your network Evaluate the security of vital records and one of a kind documents Business Interruption Insurance (do you have enough and the right coverage) Legal Considerations
RECOVERY ANALYSISDETERMINE RECOVERY STRATEGIESAlternate site arrangements Communications and network equipment Unique and/or irreplaceable equipment Resources: staff, operations support, office supplies, life support (food, water, shelter) Emergency relocation costs Disaster restoration contracts Unique and/or irreplaceable equipment Environmental and off-site requirements Identification and suspension of non-critical functions or tasks Implementing manual processing functions and tasks (is this realistic in the aftermath?)Recovery facilities should be at least 30-60 miles away from the primary siteConsider different power grids and telecom points of presence
RECOVERY ANALYSIS DETERMINE RECOVERY STRATEGIESUse internal methods when possible - Use your own facilities firstAlternate site arrangements• Vendor Hot Site, Co-location Facilities, Company Owned Hot Site, Mobile Facilities, Managed Services• Service Bureau, Office or Warehouse Space, Reciprocal Agreement, Equipment Leasing, Drying Companies and Emergency Cleaning Companies• Cold Site, Warm Site• Work Area Recovery (Call Centers, Mail Room, Specialized Equipment)• Networking and Telephone Considerations• Continuous and High-Availability• Mirroring, Replication, Clustering, Virtualization• E-Vaulting, Disk to Disk (SAN, IP SAN, NAS, ATA)• Grid Technology - supports distributed processing connecting multiple organizational sites, devices and platforms transparently, Grid is designed to assist in recovery from system failures. Cloud Computing.
Plan for Proper Decisions“If you don’t know where you’re going, you’re liable to end upsomeplace else” - Yogi Berra
Business Continuity Planning Plans Must be DOCUMENTEDInvisible Plans dont work
Business Continuity FAMILY FIRSTPEOPLE RECOVERFROM DISASTERS NOT COMPUTERS!
Business Continuity MethodsDeveloping the Business Continuity Plan Brings the research, analysis, strategies, procedures and recovery team assignments together Tasks managed and controlled at the Command Center location Contains recovery team(s) information Details the entire emergency response/crisis management process Contains contact information and notification procedures Details tasks and responsibilities Further identification of critical operations, functions and/or computer applications and how they will be recovered Specify business process recovery and restore requirements Specify software recovery and hardware configuration requirements Specify off-site storage location for your data and vital documents
Business Continuity Methods Developing the BCP (cont.) Detail recovery task sequence and functional interdependencies Identify everything that might be needed to perform part of the process: teams of people, equipment, transportation, support items, support providers, etc. Contain all procedures that might be used in the recovery process Contain a list of all vendors, service providers you will need to support your recovery strategies Contains a list of critical customers to contact Management Succession Contain standard forms (POs, Blank Checks, Travel Advances etc.), supplies and documents Moving from Disaster Mode to Normal
Business Continuity Methods WHAT DOES A PLAN LOOK LIKE? TABLE OF CONTENTS PAGE #Charter 1EXECUTIVE MANAGEMENT TEAM 3Definition of Team Members and Recovery Plan Responsibilities 4Financial TEAM 5BUSINESS CONTINUITY TEAM TASKS AND ASSIGNMENTS 6TEAM TASKS 7LAST MINUTE PREPARATION PHASE – STEP 1 7EVALUATION PHASE – STEP 2 9ACTIVATION PHASE – STEP 3 13RECOVERY CENTER START-UP PHASE – STEP 4 15RESTORATION/MOVING BACK PHASE – STEP 5 18Disaster Recovery Contact List 19RESOURCES (SEE ATTACHED SECTION)
Business Continuity MethodsTABLE OF CONTENTS PAGE # Command Center Guidelines 1Personnel Notification Guidelines and LocationNotification Guidelines 4Personnel Notification Control Log 5Emergency Telephone Numbers 6 911 CALL INSTRUCTIONS 8EMERGENCY EVENT PROCEDURES 9EVALUATION CHECKLISTS 10Declaration of Disaster (Press Release Sample) 19EMPLOYEE LOCATION LOG 20TRAVEL REQUEST FORM 21Progress Log (Used to prepare daily status reports) 23Purchase Order Forms 24RECOVERY NEEDS 25EVACUATION MAP 26CRITICAL FUNCTION PRIORITIES RECOVERY MATRIX 27MINIMUM RESOURCES REQUIRMENTS 28
Business Continuity MethodsBUSINESS CONTINUITY TEAM TASKS AND ASSIGNMENTS Initial Notification Provide Team Member Personnel Information Team Leaders to: Call and/or Assemble and Brief Team Members Deploy Teams to Alternate Facilities or have the Team Members Stay at Home Teams working from home Teams Implement Recovery Plans Operate In Crisis Mode Coordinate Recovery Actions Status Reports and Periodic Briefings (TBD) Salvage and Restoration Return Back-to-Facility/Transition Planning Post Incident Review Develop Lessons Learned Write After Action Report Update Recovery Plans
Business Continuity Methods Develop specific tasks for your office to followncident Management Team Tasks If the incident calls for an evacuation, Ensure that an orderly evacuation is taking place; Evacuate to the pre-determined meeting location; Take control of the response; Activate Incident Command Center, if required; Daytime: Assemble team at pre-determined location Daytime Primary Assembly Location: Daytime Secondary Assembly Location: Nighttime: Contact team members by telephone; Nighttime Primary Assembly Location: Nighttime Secondary Assembly Location: Set up a command and control center that can establish liaison with emergency responders, customers, the media,employees and their families, suppliers, etc.; Assemble Incident Assessment Team; Determines extent of damage from Damage Assessment Team reports;
Business Continuity Methods LAST MINUTE PREPARATION PHASE – STEP 1 1.1 Person Uncovering an IncidentIf you become aware of a potential incident within the facility: Perform all appropriate emergency notification actions (e.g. sound fire alarm, etc.). Notify Local Emergency Responder with the following information:•Your name;•Description of incident;•Preliminary report of damages and injuries;•Information regarding any attempted or actual notification contacts;•Phone number and location where you can be reached. Department Emergency Evacuation 1.2Team Leader NotificationIf in the building during the incident: Determine if equipment shutdowns are required. Contact all affected areas. Re-confirm NO ONE except Executive Management Team and/or Public Affairs is to talk with the media.Note: Team Leader will notify the Executive Management Team of function(s) shutdown.
Business Continuity MethodsProcedures Lists to be Developed and Maintained for allDepartmental Business Continuity PlansOffsite Storage Retrieval ProceduresDepartment Operational ProceduresEmergency Procurement ProceduresVendorsEmployee Names/Addresses/Phone NumbersDepartment Equipment ListJob DescriptionsResource Requirements
Business Continuity Testing Plan Exercising - The Plan is AliveBefore any recovery plan can be considered complete, it must bevalidated. Plan testing is a “practice recovery;” it allow you tovalidate the strategies, procedures and recovery team structuresdocumented in your recovery plan. Plan testing normally consistsof a mock disaster scenario or moving your critical applications toan alternate facility. We recommend that your recovery teamsparticipate fully in the plan rehearsal, to validate team structuresand responsibilities.
Business Continuity Program Lifecycle and Maintenance Plan Review Component Testing IntegratedUpdate Plan Standards Planning and Testing Awareness Exercise Training Plan Perform Maintenance Schedule
Discussion – Thank YouThank you for attending this presentation Continuity Solutions, Inc. 5900 Roche Drive Columbus, Ohio 43229 (614)-569-3292 www.csigroup.cc