Business continuity at_northrop_grumman


Published on

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Business continuity at_northrop_grumman

  2. 2. Agenda Brief History about the company About ship systems sector (Katrina strikes and the damages) Starting of Recovery Restoring the IT infra Planning recovery and learning from disaster Lessons Learnt from Northrop Grumman
  3. 3. Brief About the company <ul><li>A $33 billion USD company with 120,000 employees worldwide </li></ul><ul><li>Provides innovative systems, products and services in aerospace, electronics, IS, shipbuilding and technical services </li></ul><ul><li>Customers include US government and commercial enterprises worldwide </li></ul><ul><li>Shipbuilding is the highest revenue earner. US’s sole designer and builder of nuclear powered aircraft carrier </li></ul><ul><li>Ship systems employees:12900 in Pascagoula & Gulfport, 7100 in New Orleans </li></ul><ul><li>Shipbuilding facilities in Pascagoula and New port News, Virginia </li></ul>
  4. 4. Hurricane Katrina <ul><li>A Category 5 hurricane with 175 miles wind speed with 200 miles gust </li></ul><ul><li>Landfall on 29 th August in Mississippi </li></ul><ul><li>30ft surge,238 causalities in MS </li></ul><ul><li>New Orleans went underwater with levees broken and pump failure </li></ul><ul><li>1577 deaths in Louisiana and massive property damages </li></ul>
  5. 5. Some pictures
  6. 6. Damages
  7. 7. Damages to Northrop Grumman <ul><li>Severe damage to the under construction ships, the shipyard </li></ul><ul><li>Power and network outage </li></ul><ul><li>1,500 PCs,200 servers, 300 printers, 600 data input devices, and hundreds of two-way radios damaged </li></ul><ul><li>Primary data centre and network equipments were gone and link with outside was cut </li></ul><ul><li>In NO, servers shut down due to air conditioner failure </li></ul><ul><li>Some means of communication still alive in NO </li></ul><ul><li>Estimated loss of Northrop Grumman was $1 billion USD </li></ul>
  8. 8. Starting of recovery <ul><li>Preparedness proved worth with less damage in data </li></ul><ul><li>Restoring human resources was primary concern </li></ul><ul><li>Information centre at Dallas was into immediate action </li></ul><ul><li>Supplies to affected area in corporate jets, helicopters and cargo planes </li></ul><ul><li>IT workers became productive after relocation </li></ul><ul><li>Arrangements with Wal-Mart and Western Union </li></ul>
  9. 9. Restoring the IT infrastructure <ul><li>Both the data center (Pascagoula and Gulfport) damaged </li></ul><ul><li>Unavailability Public communication & other infrastructural facilities </li></ul><ul><li>Switching stations, cellular antennas and other essential elements damaged </li></ul><ul><li>Satellite communication equipment used for initial damage assessment </li></ul>
  10. 10. Reconnecting the yards <ul><li>Restoring communication </li></ul><ul><ul><li>Satellite phone was initially ; line-of-sight access to satellites </li></ul></ul><ul><ul><li>Satellite dishes installed to provide bandwidth for voice and data VoIP </li></ul></ul><ul><ul><li>Pascagoula data center could not be rebuilt ; reopening of the yards </li></ul></ul><ul><ul><li>Walkie Talkies replaced the two way radios previously used for within yard communication </li></ul></ul>
  11. 11. Dallas : New Datacente r <ul><ul><li>Procurement Challenge : Replacing 1500 desktop , 200 servers and communication infrastructure </li></ul></ul><ul><ul><li>Ongoing consolidation some applications were migrated to Dallas </li></ul></ul><ul><li>Space available ; personnel and equipment </li></ul><ul><li>Employee from Pascagoula relocated to Dallas within 2 days. </li></ul><ul><li>Restoration challenges </li></ul><ul><ul><li>Hardware : Hitches such as relocation of server required taking disrupting business. </li></ul></ul><ul><ul><li>Incompatibility between hardware and software environment. </li></ul></ul><ul><ul><li>Inaccessibility of SAD </li></ul></ul>
  12. 12. Restoring Data and Applications <ul><li>No data lost due to the IT disaster preparedness. </li></ul><ul><li>Data backups @ Iron Mountain & Dallas </li></ul><ul><li>Challenge : Prioritizing application recovery due to lack of properly documented business continuity plan </li></ul><ul><li>Change in application criticality e.g. e-mail & communications infrastructure </li></ul><ul><li>DDG 1000’s : Migrated to Dallas & brought up in 6 days. </li></ul>
  13. 13. Planning, Recovering and Learning from the Disaster <ul><li>Recovery (within 2 weeks) </li></ul><ul><ul><li>New Data Center </li></ul></ul><ul><ul><li>Both Shipyards had connectivity </li></ul></ul><ul><ul><li>All employee’s and their families found </li></ul></ul><ul><ul><li>Agility, teamwork and Leadership were the key </li></ul></ul>
  14. 14. Disaster Preparedness <ul><li>“ The best form of crisis management is preparedness” </li></ul><ul><li>Most Organizations </li></ul><ul><ul><li>Prepare for specific domains </li></ul></ul><ul><ul><li>Having greatest impact on their operations </li></ul></ul><ul><ul><ul><li>E.g. Financial institutions preparing for IT failure, hospitals for pandemics </li></ul></ul></ul><ul><li>Alternative </li></ul><ul><ul><li>Broad Spectrum – Economic, Information, Physical, Human resources </li></ul></ul><ul><ul><li>Portfolio Approach – One in each portfolio </li></ul></ul>
  15. 15. COBIT vs. ITIL
  16. 16. Advantages : IT Governance COBIT (Control Objectives for Information and Related Technology) and ITIL (IT Infrastructure Library) include sections on disaster preparedness and business continuity planning. Understood to both IT and business managers. Supports Audit Process. Generic objectives and measurements Excellent guidelines for establishing IT disaster preparedness
  17. 17. Dis-advantages of Frameworks <ul><li>Limited Guidance to operational level </li></ul><ul><li>Emphasis on IT leaves out planning parameters outside technical realm </li></ul><ul><li>Lot of assumptions </li></ul><ul><li>Less direction on recovery strategies </li></ul>
  18. 18. Learning from Disaster : Case for Post Disaster Analysis <ul><li>Prepare for unanticipated Crisis </li></ul><ul><li>Nature of disaster shifts over time </li></ul>No post recovery analysis COBIT or ITIL do not offer guidance
  19. 19. Lessons Learned <ul><li>Keep Data and Data Centers Out of Harm’s Way </li></ul><ul><li>Don’t Assume the Public Infrastructure Will Be Available </li></ul><ul><ul><li>As NGC adapted to non availability of infra on the fly rather than reference plans. </li></ul></ul><ul><ul><li>Its leadership in crisis is an inspiration to future planners. Today, its disaster preparedness plans include the vulnerability of the public infrastructure. </li></ul></ul><ul><ul><li>As a result, extra satellite phones are now in secure locations, and satellite dishes can be safely sheltered and then redeployed after disruptive events. </li></ul></ul>
  20. 20. Lessons Learned <ul><li>Plan for Civil Unrest </li></ul><ul><ul><li>Management decided to bring in personnel from outside the area to secure the facility </li></ul></ul><ul><ul><li>Disaster preparedness can include contracting for such services and planning how security personnel will be notified, transported, fed, and supported during the period of threat. </li></ul></ul><ul><li>Assume Some People Will Not Be Available </li></ul><ul><ul><li>Fortunately, NGC had employees who were called upon to start recovery operations </li></ul></ul><ul><ul><li>Employees with little computing background were also deployed </li></ul></ul><ul><ul><li>Post Katrina, NGC has established backup role takers for all key personnel. </li></ul></ul>
  21. 21. Lessons Learned <ul><li>Leverage Your Suppliers as Critical Team Members </li></ul><ul><ul><li>Implementing a contingency plan is closely linked to suppliers’ capabilities. </li></ul></ul><ul><ul><li>Assessing damage quickly and getting orders to suppliers can reduce bottlenecks in supply chain, particularly if other firms are also clamoring for replacement equipment. </li></ul></ul><ul><ul><li>Think of it had Katrina hit Los Angeles? </li></ul></ul><ul><li>Expect the Unexpected </li></ul><ul><ul><li>Build a “crisis portfolio” by preparing for at least one disaster in each crisis category of disasters. </li></ul></ul><ul><ul><li>No economic sense to plan for all type of disasters </li></ul></ul><ul><ul><li>Larger the firm - > More Risk prone - > More risks come outside the scope of BCP </li></ul></ul>
  22. 22. Lessons Learned <ul><li>Get Prepared </li></ul><ul><ul><li>Plan processes to deal with unplanned events (Use COBIT or ITIL) </li></ul></ul><ul><ul><li>Identify crisis teams in multiple locations, define roles, and identify likely role holders and backups. </li></ul></ul><ul><ul><li>Train and test scenarios. </li></ul></ul><ul><ul><li>List supplies most likely to be needed, and identify suppliers </li></ul></ul><ul><ul><li>Agree beforehand on exceptions to procurement policies during states of emergency. </li></ul></ul><ul><ul><li>Include in the contact list the telephone numbers, e-mail addresses, and other contact information of people outside the potential disaster area who might assist in locating missing personnel. </li></ul></ul><ul><ul><li>Inform employees how to call in, before a disaster hits </li></ul></ul><ul><ul><li>IT recovery document should be simple and consistent help backup teams get in place quickly. </li></ul></ul><ul><ul><li>Consider simulating a disaster, evaluating the responses, and modifying your plans accordingly </li></ul></ul>
  23. 23. Lessons Learned <ul><li>Establish a Strong Leadership Position </li></ul><ul><ul><li>Reacting to the situation at hand than executing a pre-defined plan. </li></ul></ul><ul><ul><li>Use the disaster recovery plan as a model. But be flexible in allowing leadership skills to transcend an inadequate plan. </li></ul></ul><ul><ul><li>Act quickly (at “hurricane speed”) and decisively. There may be no time for a committee discussion. Decision makers may have to fall back on their instincts and leadership experience </li></ul></ul><ul><ul><li>Sharing facts stimulates the flow of feedback from disparate, disaster-affected key constituents, which, in turn, increases the quality of information for decision making. </li></ul></ul>
  24. 24. Lessons Learned <ul><li>Empower Decision Makers on the Team </li></ul><ul><ul><li>Often, pre-established decision hierarchies dissolve in crises. </li></ul></ul><ul><ul><li>In addition, leadership becomes more centralized and more decentralized, simultaneously. </li></ul></ul><ul><ul><li>With first-hand knowledge and the necessity to execute, people at the disaster scene often must exceed their prior authorities and operational scope to move the recovery effort forward. </li></ul></ul><ul><li>Exploit Fresh Start Opportunities </li></ul><ul><ul><li>Opportunity to upgrade ? </li></ul></ul><ul><ul><li>Katrina accelerated consolidation of DCs </li></ul></ul><ul><ul><li>Resistance to Upgrades evanporated </li></ul></ul><ul><ul><li>New technologies were brought in… </li></ul></ul>
  25. 25. <ul><li>Thank You </li></ul>