Business continuity at_northrop_grummanPresentation Transcript
BUSINESS CONTINUITY AT NORTHROP GRUMMAN ANSHUMAN DEEPTI PARTH SOUMYA (GROUP 7)
Agenda Brief History about the company About ship systems sector (Katrina strikes and the damages) Starting of Recovery Restoring the IT infra Planning recovery and learning from disaster Lessons Learnt from Northrop Grumman
Brief About the company
A $33 billion USD company with 120,000 employees worldwide
Provides innovative systems, products and services in aerospace, electronics, IS, shipbuilding and technical services
Customers include US government and commercial enterprises worldwide
Shipbuilding is the highest revenue earner. US’s sole designer and builder of nuclear powered aircraft carrier
Ship systems employees:12900 in Pascagoula & Gulfport, 7100 in New Orleans
Shipbuilding facilities in Pascagoula and New port News, Virginia
A Category 5 hurricane with 175 miles wind speed with 200 miles gust
Landfall on 29 th August in Mississippi
30ft surge,238 causalities in MS
New Orleans went underwater with levees broken and pump failure
1577 deaths in Louisiana and massive property damages
Damages to Northrop Grumman
Severe damage to the under construction ships, the shipyard
Power and network outage
1,500 PCs,200 servers, 300 printers, 600 data input devices, and hundreds of two-way radios damaged
Primary data centre and network equipments were gone and link with outside was cut
In NO, servers shut down due to air conditioner failure
Some means of communication still alive in NO
Estimated loss of Northrop Grumman was $1 billion USD
Starting of recovery
Preparedness proved worth with less damage in data
Restoring human resources was primary concern
Information centre at Dallas was into immediate action
Supplies to affected area in corporate jets, helicopters and cargo planes
IT workers became productive after relocation
Arrangements with Wal-Mart and Western Union
Restoring the IT infrastructure
Both the data center (Pascagoula and Gulfport) damaged
Unavailability Public communication & other infrastructural facilities
Switching stations, cellular antennas and other essential elements damaged
Satellite communication equipment used for initial damage assessment
Reconnecting the yards
Satellite phone was initially ; line-of-sight access to satellites
Satellite dishes installed to provide bandwidth for voice and data VoIP
Pascagoula data center could not be rebuilt ; reopening of the yards
Walkie Talkies replaced the two way radios previously used for within yard communication
Dallas : New Datacente r
Procurement Challenge : Replacing 1500 desktop , 200 servers and communication infrastructure
Ongoing consolidation some applications were migrated to Dallas
Space available ; personnel and equipment
Employee from Pascagoula relocated to Dallas within 2 days.
Hardware : Hitches such as relocation of server required taking disrupting business.
Incompatibility between hardware and software environment.
Inaccessibility of SAD
Restoring Data and Applications
No data lost due to the IT disaster preparedness.
Data backups @ Iron Mountain & Dallas
Challenge : Prioritizing application recovery due to lack of properly documented business continuity plan
Change in application criticality e.g. e-mail & communications infrastructure
DDG 1000’s : Migrated to Dallas & brought up in 6 days.
Planning, Recovering and Learning from the Disaster
Recovery (within 2 weeks)
New Data Center
Both Shipyards had connectivity
All employee’s and their families found
Agility, teamwork and Leadership were the key
“ The best form of crisis management is preparedness”
Prepare for specific domains
Having greatest impact on their operations
E.g. Financial institutions preparing for IT failure, hospitals for pandemics
Broad Spectrum – Economic, Information, Physical, Human resources
Portfolio Approach – One in each portfolio
COBIT vs. ITIL
Advantages : IT Governance COBIT (Control Objectives for Information and Related Technology) and ITIL (IT Infrastructure Library) include sections on disaster preparedness and business continuity planning. Understood to both IT and business managers. Supports Audit Process. Generic objectives and measurements Excellent guidelines for establishing IT disaster preparedness
Dis-advantages of Frameworks
Limited Guidance to operational level
Emphasis on IT leaves out planning parameters outside technical realm
Lot of assumptions
Less direction on recovery strategies
Learning from Disaster : Case for Post Disaster Analysis
Prepare for unanticipated Crisis
Nature of disaster shifts over time
No post recovery analysis COBIT or ITIL do not offer guidance
Keep Data and Data Centers Out of Harm’s Way
Don’t Assume the Public Infrastructure Will Be Available
As NGC adapted to non availability of infra on the fly rather than reference plans.
Its leadership in crisis is an inspiration to future planners. Today, its disaster preparedness plans include the vulnerability of the public infrastructure.
As a result, extra satellite phones are now in secure locations, and satellite dishes can be safely sheltered and then redeployed after disruptive events.
Plan for Civil Unrest
Management decided to bring in personnel from outside the area to secure the facility
Disaster preparedness can include contracting for such services and planning how security personnel will be notified, transported, fed, and supported during the period of threat.
Assume Some People Will Not Be Available
Fortunately, NGC had employees who were called upon to start recovery operations
Employees with little computing background were also deployed
Post Katrina, NGC has established backup role takers for all key personnel.
Leverage Your Suppliers as Critical Team Members
Implementing a contingency plan is closely linked to suppliers’ capabilities.
Assessing damage quickly and getting orders to suppliers can reduce bottlenecks in supply chain, particularly if other firms are also clamoring for replacement equipment.
Think of it had Katrina hit Los Angeles?
Expect the Unexpected
Build a “crisis portfolio” by preparing for at least one disaster in each crisis category of disasters.
No economic sense to plan for all type of disasters
Larger the firm - > More Risk prone - > More risks come outside the scope of BCP
Plan processes to deal with unplanned events (Use COBIT or ITIL)
Identify crisis teams in multiple locations, define roles, and identify likely role holders and backups.
Train and test scenarios.
List supplies most likely to be needed, and identify suppliers
Agree beforehand on exceptions to procurement policies during states of emergency.
Include in the contact list the telephone numbers, e-mail addresses, and other contact information of people outside the potential disaster area who might assist in locating missing personnel.
Inform employees how to call in, before a disaster hits
IT recovery document should be simple and consistent help backup teams get in place quickly.
Consider simulating a disaster, evaluating the responses, and modifying your plans accordingly
Establish a Strong Leadership Position
Reacting to the situation at hand than executing a pre-defined plan.
Use the disaster recovery plan as a model. But be flexible in allowing leadership skills to transcend an inadequate plan.
Act quickly (at “hurricane speed”) and decisively. There may be no time for a committee discussion. Decision makers may have to fall back on their instincts and leadership experience
Sharing facts stimulates the flow of feedback from disparate, disaster-affected key constituents, which, in turn, increases the quality of information for decision making.
Empower Decision Makers on the Team
Often, pre-established decision hierarchies dissolve in crises.
In addition, leadership becomes more centralized and more decentralized, simultaneously.
With first-hand knowledge and the necessity to execute, people at the disaster scene often must exceed their prior authorities and operational scope to move the recovery effort forward.