The Future Of DoD Metrics
Upcoming SlideShare
Loading in...5

The Future Of DoD Metrics



Software metrics, including Lean Six Sigma, used in the Joint Strike Fighter Department of Defense Program

Software metrics, including Lean Six Sigma, used in the Joint Strike Fighter Department of Defense Program



Total Views
Views on SlideShare
Embed Views



2 Embeds 3 2 1



Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

The Future Of DoD Metrics The Future Of DoD Metrics Document Transcript

  • The Future Department of Defense Metrics by James W. Didier SE 468-701 Professor Dennis Mumaugh November 23, 2009
  • Future of DoD Metrics 2 Table of Contents Abstract 3 Introduction 3 History Department of Defense 4 Lean Six Sigma 5 Current DoD Metrics & Methods Joint Strike Fighter Program 6 Software 7 Manufacturing 10 JSF Summary 11 Future Challenges 11 Recommendations 13 Conclusions 13 Glossary 15 Cited References 18 Non-Cited References 18 Diagrams 20
  • Future of DoD Metrics 3 ABSTRACT The purpose of this research is to evaluate how Lean Six Sigma is currently being used by the Department of Defense (DoD) and how this methodology might evolve to meet future needs. You will be introduced to the Department of Defense, Six Sigma, and Lean development. To study Lean Six Sigma implementation, within the Department of Defense, I examine a current defense contract, signed by Lockheed Martin, called the Joint Strike Fighter F-35 program. This is one of the largest and most complex international arms sales in history. Production of the F-35 will rush in a revolutionary manufacturing process responsible for creating the most sophisticated 5th generation stealth strike fighter the world has ever seen. INTRODUCTION In today’s world, the Department of Defense (DoD) projects are inherently becoming more expensive. These projects involve ever increasing risk due to their size and complexity. Now software intensive systems involve more lines of code (LOC) and incorporating never seen before system integration. These systems use a mix of commercial off the self (COTS) software and defense applications. These projects will involve an increasing number of stakeholders to control costs. Requirements may be shared among numerous government agencies and military services. As we look into the future, metrics and measurements will have an increasingly important role to play. This research will be broken down into the following main categories. 1) An introduction to the Department of Defense, Six Sigma, and Lean development. 2) Lean Six Sigma implementation within the Joint Strike Fighter Program. 3) Describe the challenges the Department of Defense will face in the years ahead. 4) How metrics should evolve to meet the future needs of the Department of Defense. I find that the Department of Defense is often one foot ahead of the corporate world and by studying current defense projects you will be better prepared to handle business ventures. The Department of Defense set standards that the corporate world improves upon. This cycle repeats and evolves. By exploring this topic, it will convey an understanding of what business needs the future may bring for metric processes.
  • Future of DoD Metrics 4 HISTORY Department of Defense The US Department of Defense (DoD) is a military establishment created in 1947, to preside over the former Department of War founded in 1789, with the purpose of managing all government agencies relating directly to US national security and military operations. World War II (WWII) proved to the US that such a department is mandatory for national security and coordinating military efforts. Figure 1 displays an old WWII poster. The DoD includes the Army, Navy, Air Force, Marines Corps, National Security Agency (NSA), Defense Intelligence Agency (DIA), Figure 1: WWII Poster. National Geospatial-Intelligence Agency (NGA), Missile Defense Agency (MDA), Defense Advanced Research Projects Agency (DARPA), and Pentagon Force Protection Agency (PFPA). The Army is responsible for land-based military operations, the Navy is responsible for sea-based military operations, and the Marines Corps is responsible for amphibious military operations by sea. The NSA is an intelligence agency responsible for the cryptanalysis of foreign communication, signals intelligence, and information systems. The DIA is an intelligence agency responsible for producing and managing timely military intelligence, espionage, and counter-intelligence. The NGA is an intelligence agency responsible for the collection, analysis, and distribution of geospatial intelligence (i.e. terrestrial mapping and imagery) for both military and national security purposes. The MDA is responsible for developing a layered military response to defend the US against nuclear ballistic missiles. The DARPA is responsible for the development, funding, or sponsorship of new leading-edge military technology. The PFPA is a government agency responsible for protecting and policing the Pentagon. The DoD also operates several schools including the National War College. The DoD is the largest military organization with a budget more than twice that of the world’s largest corporation and has a larger population than 3rd world countries1. The DoD awards defense contracts to regulated companies that provide products to the US military. For this reason, the research will focus on the military services since it consumes the majority of funding and is the most complex. The type of project chosen for the acquisition of major weapon systems and/or software-intensive systems is based on a DoD risk assessment. DoD assessments identify the risks to US interests and how to mitigate this risk. One of the most famous risk assessments is called the Quadrennial Defense Review that can be found at the DoD web site at This is the context in which Lean Six Sigma is applied.
  • Future of DoD Metrics 5 Lean Six Sigma Lean Six Sigma, conceived in 1988, combines Six Sigma quality metrics with Lean development. Lean development was created from automobile manufacturing. Lean development has roots in the Toyota Production System. Its main original emphasis was on eliminating seven wastes in over-production, unnecessary processing, and waiting. Other wastes include latent skill, danger, poor information, material, and breakdown. With these wastes removed manufacturing becomes faster and flexible while Figure 2: WWII Poster. mitigating risk. Figure 2 displays the attitude incorporated by Lean. The efficiency gained by optimizing work flow and automation improves overall customer value. Pull systems are those that demand something and push systems supplies something. These concepts are incorporated into the workflow analysis. Lean development has also been applied to IT known as Lean IT. Lean IT tries to eliminate wastes pertaining to defects, over- production, waiting, non-value processing, transportation, inventory, motion (firefighting repeated issues), and employee problems. Lean can also be applied to any service and this is known as Lean Service. Learn Service is the application of Lean principles to any service. Service wastes pertain to delay, duplication, unnecessary movement, unclear communications, incorrect inventory, an opportunity lost to retain or win customers, and errors in service transactions. This applies the concepts of value demand and failure demand. Value demand is when the customer demands service and failure demand is the failure to do something right for the customer. The application of Lean services into a workflow causes continuous self improving project management that reinforces the organization to see things from the customer’s point-of-view. Six Sigma, conceived in 1986, is a business management strategy designed to use quality management and statistical models to improve management processes and eliminate defects. The goal is to achieve 3.4 defective parts per million opportunities. This methodology asserts that there should be continuous effort to achieve stable and predictable results from any process. These processes have characteristics that can be measured, analyzed, improved, and controlled. Senior leadership must support these principles. In addition, there other principles that set Six Sigma apart from the other methodologies. Six Sigma focuses on achieving measurable and quantifiable financial returns from any project. Senior leadership must be passionate and support Six Sigma which incorporates a ranking system that allows highly ranked employees to management projects. Decisions must be made using verifiable data and statistical modeling. There are various project management methodologies. One states that you must define goals, measure key aspects and define behavior, analyze data correlations to verify cause-and- effect, improve data analysis, and control target deviations before they result in defects. A branch of Six Sigma, called Design for Six Sigma, is used for complex products and
  • Future of DoD Metrics 6 applied to Software Engineering. Here the project management involves defining goals, measuring critical to qualify characteristics, analyzing, designing details, and verifying. This methodology is incorporated into the Software Development Life Cycle. The criticisms of Six Sigma are that it is too narrowly defined to fix an existing process and may not improve profitability. This leads to the creation of Lean Six Sigma. Lean Six Sigma, developed in the late 1990s, combines quality management of Six Sigma and speed of Lean development. "Lockheed Martin recognized that our business support processes have as much opportunity for improvement as our design and build areas. By applying Lean process speed and Six Sigma quality tools to marketing, legal, contract administration, procurement, etc. we have created a competitive advantage.... The lessons learned and practical case studies contained in Lean Six Sigma for Service provide a road map which can create great value for customers, employees and shareholders."--Mike Joyce, Vice President, Lockheed Martin Operational Excellence CURRENT DOD METRICS & METHODS Joint Strike Fighter Program The best way to look at how Lean Six Sigma is being implemented is to look at a current DoD project. Currently the biggest DoD project is called the Join Strike Fighter program. The System Development and Demonstration contract was awarded to Lockheed Martin on October 26th, 2001 after beating out Boeing. In April 2009, the Pentagon contemplates buying around 2,443 Figure 3: F-35A. aircraft. BAE Systems and Northrop Grumman are major partners. The Joint Strike Fighter is a program to development a 5th generation stealth strike fighter for the US Navy, US Marine Corps, US Air Force, United Kingdom, Italy, Netherlands, Canada, Turkey, Australia, Norway, Denmark, Israel, and Singapore. The Joint Strike Fighter, F-35 Lightning II, will have 3 variants. The US Air Force wants a conventional takeoff/landing (CTOL) F-35A version that has a gun and can physically sustain 9 g-forces. This is seen in figure 3. The US Marine Corps and the United Kingdom want a short takeoff/vertical landing (STOVL) F-35B version. This means it can hover in place and land without a runway. The US Navy wants a carrier (CV) version that has a reinforced structure that can take the brutality of landing on an aircraft carrier. The production methods are stated to be revolutionary creating one 5th generation strike fighter a day. In February of 2008, The Government Accountability Office (GAO) created congressional report GAO-08-294 that discusses the best practices called Increased Focus on Requirements and Oversight Needed to Improve DoD’s Acquisition Environment and Weapon System Quality. Problems related to quality impacted 11 DoD weapon systems
  • Future of DoD Metrics 7 reviewed causing billions in cost overruns, year-long delays, and decreased capabilities3. Officials stated the importance of quality and how prime contractors are improving quality. This includes the use of Six Sigma. These companies must insure requirements are achievable, manufacturing can consistently produce a high quality product with low variability, and all suppliers’ shipments are of quality. For example, both Cummins and Kenworth use Six Sigma. Both organizations require their suppliers to use Six Sigma as well. Here lies one of the problems with the F-35. Lockheed does use Lean Six Sigma. They also inspect all parts from their international suppliers. Six Sigma and proper CMM levels may not be mandatory for all suppliers. The time it takes to receive new parts causes delays within the schedule. The likelihood that this will happen increases due to the translation of business requirements for international suppliers. Programs like the Joint Strike Fighter are political as well. The DoD promotes using international suppliers to cut costs and negotiate the adoption of the F-35 abroad. Critical design reviews are used early in the research and development of the aircraft. The second GAO critique is on the change in requirements that lead to a redesign. Once all three military services agreed to the requirements one service wanted to increase the size of the weapon bay allowing a larger Figure 4: F-35 Manafacturing. payload. Stealth fighters carry their weapons internally to reduce the chance of being spotted by radar. The result increased the F-35’s weight and the STOVL version was too heavy. This caused a major redesign to reduce weight. This example of requirement creep, within a multinational defense project, had disastrous results for one of the stakeholders. One of the strategic requirements for Joint Strike Fighter Program is to create affordable stealth aircraft. Other aircraft, such as the F-22A Raptor, is too expensive to buy in quantity. Here we begin to see the limitations of Lean Six Sigma. The politics of international arms sales and overzealous changes in requirements begin to reduce the chances of achieving affordable stealth aircraft that can be bought in quantity. Figure 4 shows the Secretary of Defense Dr. Robert Gates as he tours the F-35 manufacturing plant while being reassured that Lean Six Sigma and the new manufacturing processes will deliver the F-35 on time. Software F-35 is a prime example of software-intensive systems due to the lines of code, number of components and integration. Figure 5 displays the F-35’s cockpit. The f-35 has an AN/APG-81 Active Electronically Scanned Array (AESA) able to simultaneously engage air-to-air and air-to-ground targets, conduct electronic warfare, and track multiple targets while searching for more. Twenty three targets, ranging up to 100 miles away, Figure 5: F-35 Cockpit. can be tracked in 9 seconds. The AESA radar will last
  • Future of DoD Metrics 8 the entire life-time of the aircraft and flight-time repairs are under 30 minutes. This radar has both active and passive modes. F-35 has synthetic aperture radar (SAR) that can map the geography, track ground vehicles, and identity the vehicles. Electro-optical targeting system (EOTS) allows identification and targeting without the use of the AESA radar. There is a digital aperture system (DAS) that allows 360 degree tracking and targeting of aircraft. Mission systems upgrades and configuration can be done with devices similar to a Personal Digital Assistant (PDA). The aircraft performs real-time maintenance checks, identifies defects, and notifies ground control so the malfunctioning part can be replaced immediately after landing. The planes have stealth monitoring systems that enable the pilot to verify if the stealth is working properly during combat. These 5th generation fighters plug in to the US net-centric warfare capabilities, within the battlespace, by use of secure digital links between other military assets. As one F-35 tracks a target the other F-35 can receive the radar picture. An F-35 can perform electronic warfare by jamming or burning out sensitive electronics. This might include navigation and flight control systems among others. All of these systems are integrated together within the plane’s software architecture and projected by the pilot’s helmet-mounted display (HMD). Figure 6 displays the HMD. The F-35, itself, is connected to all other military systems within the battlespace. This is called net-centric warfare. For example, an F-35’s radar is damaged in battle so it may use satellite imaging to compensate. When you look at the lines of code (LOC) from the F-15E Strike Eagle (introduced in 1986) to the F-22A Raptor (introduced in 2005) and ending with the F-35 Lightning II (introduced in 2012) they increase exponentially. I can’t give exact LOC but the F-35 has over triple the amount of code that the f-22 has. Lean Six Sigma will play a big role in this area. The systems are too large and complicated to measure everything. Yet you need to track and fix defects. One defect can have a devastating effect. In 2007, some of the F-22 systems went off line when flying over the International Date Line. This could have devastating effects during a time of war. One defect can cause multiple system errors due to the data fusion involved. The difficulty in military acquisitions is that the technology is usually not mature but is estimated to be before production. An aircraft take years to research, design, and build. Often by the time of production the aircraft’s technology is not cutting edge or the threats that this aircraft was designed to counter Figure 6: HMD. have changed. It is a guessing game. So the DoD does studies trying to estimate if the technology will be feasible for integration within the later phases of aircraft development. This creates risk since aircraft must be delivered on schedule. F-35s must be completed on time to replace per-existing legacy aircraft that are retiring. The rate of F-35 production must match the rate of retiring aircraft or else the gap created might be
  • Future of DoD Metrics 9 considered a weakness that any enemy may exploit. GAO has criticized Lockheed Martin for their lack of risk management, schedule slippage, and cost overruns. Lockheed Martin is hoping to make up time within their manufacturing phase. This is where Lean Six Sigma can have a big impact. In research and development, flying 737 CAT-bird, was used to test avionics to speed up development and lower costs. During the move to low-rate production computers and test harnesses will be used to test all devices. An estimated 19 million lines of code will be used to support JSF production. This does not include F-35 system’s software. To meet these challenges Lockheed Martin deployed Serena Dimensions to track and reinforce several hundred processes within software development and change management. This will enforce proper procedures for critical systems that will result in saving both time and cost. This will help in reporting, auditing, and securing sensitive information. This will ensure that development corresponds to International Traffic in Arms Regulations (ITAR). Remember that there are 4 versions of the aircraft and multiple international partners. Each international partner has a rank determining what type of systems and security access they get. The system must track all software ensuring the right software is implemented on the right aircraft. Plus later versions of software will be associated with different “blocks.” A “block” is essentially the next version of the aircraft. F-35A block I systems at low-rate production might differentiate from F-35A block II systems 5 years later during peak production. These blocks must go through the Software Development Lifecycle and get certified as combat ready. Custom reports can be generated with ease for tracking and analysis purposes. Changes, history, and metadata will be maintained. You must track data but you must also make sure the data can only be accessed by certain personnel. This information is classified so protection against both industrial and military espionage is mandatory. Security is a big issue. The usability of the software and tools are easy to teach throughout the 12 years of F-35 manufacturing. Custom interfaces can be created for specific user groups. These processes decrease management overhead, limit waste, and incorporate lean development principles. Developers can self monitor error Figure 7: Net-Centric Warfare Integration. checking and reviews. The software architecture is performance based and extendable. Figure 7 displays some of the communication that must take place in net-centric warfare. The F-35 must be a fault tolerant system that incorporates some commercial-off-the-self (COTS) components to reduce cost. Any COTS software must comply with DoD standards. This testing is due to the high g-forces experienced and the possibility of electronic warfare. There is a lawsuit, by former Lockheed Martin Corporation employees, stating that the software is unsafe. The lawsuit states that government procedures are not being followed. There is also another lawsuit stating the same thing about Lockheed’s F-22A software. The former employees claim that the Lockheed Software Management Team (SMT) and
  • Future of DoD Metrics 10 the JSF Program Office (JPO) misled, concealed, and interfered with software safety compliance to ensure funding. Lockheed Martin does have waivers when software needs to be tailored but none are associated with this case. These former employees said that they were reprimanded when they continually approached management about this issue. Both Safety Evidence Assurance Level (SEAL) and Software Engineering Institute Capability Figure 8: F-35 Manufacturing. Maturity Model (SEI/CMM) are stated as not being followed. SEAL is a level of quality assurance needed for a specific system. Flight control systems are defined as SEAL 1 meaning critical. The lawsuit stated the F-35 flight control systems failed in simulated flight on November 3rd 20044. To put this into context let me added the events that made the news recently. In 2007, the F-22 lost navigation systems when crossing the International Date Line. In March 2009, a pilot was killed when an F-22 crashed. In May 2009, this lawsuit was filed. Lockheed Martin maintains that all software complies with SEI/CMM Level 4. Standards are created to log, track, and fix defects. This is dictated by the software development plan. The documents included are the Software Product Evaluation (SPE) data, unit and formal test cases, reports, and metrics. Defects are maintained in a database called Software Problem Anomaly Reports (SPAR) that the government can access. Lockheed Martin claims that these are disgruntled employees and that the software development plans (SDPs) call for a metric data review once a month. Manufacturing The GAO stated that the F-35 is well above the original estimated cost. Lockheed Martin believes it can make up much of the time in manufacturing. Lockheed states by 2016 the plant will produce roughly one F-35 per day. Secretary of Defense Dr. Robert M. Gates personally viewed the F-35 production line on August 31, 2009 in Fort Worth Texas. Now the F-35 is in low-rate production as testing continues. This is being done to ensure the on time delivery of the aircraft as current legacy fighters are being retired. This is quite a risky move since testing has not been completed and the cost per unit increases. Lockheed Martin is adopting an assembly line system of aircraft production in their brand new F-35 facility at Fort Worth Texas. Assembly line aircraft manufacturing has not been done since WWII. Figures 8 and 9 show the F-35 assembly line manufacturing. The facility is so big the employees have to drive or Figure 9: F-35 Assembly Line.
  • Future of DoD Metrics 11 bike to the other end of the plant. All three aircraft variants will be built here. The workforce is estimated to reach 3,500 at the peak of full-rate production. Lockheed Martin will both test and produce fighters on the same assembly line to decrease costs and increase productivity. Each workstation can accommodate all the versions of the F-35 to save costs. Yellow platforms surround the aircraft as it constantly moves throughout the plant. Big blue towers extend to provide cooling, electrical, and hydraulic power so aircraft systems can be tested when the aircraft is still being built. There will be a total of 12 moving platforms. Most tools and components will have 3D schematics. The aircraft will be built inside out. Most planes are built by inserting components into an airframe which can damage both the component and the airframe. High level of automation will be used to drill holes, assemble computer parts, and even choose the right fastener during construction. Employees will log in, be given daily tasks, and then be given tools. Since this is done on login all tools are accounted for and inventory is always up-to-date. The process can even account for the wear-and-tear in the inventory. There will be a highly advanced supply chain that will inspect for quality and ensure timely shipments. Figure 10 displays all the current and potential F-35 customers. JSF Summary Reportedly there are accusations that the DoD is not managing the F-35 program correctly and Lockheed Martin is not managing risks appropriately. This is leading to cost and schedule overruns. The Earned Value Management System data can be misleading since some data may not be recorded. There is a lawsuit, GAO reports, and news events that support this position. There were two major F-22 system failures and one of them killed the pilot in 2009. Lockheed Martin may even be using its management reserves to hide the performance of subcontractors. There are several areas that Lockheed Martin may not be DoD compliant. The essential thing about Lean Six Sigma is that management must support it. In my opinion, Lockheed Martin’s management supports covering up data while the JPO looks the other way. Lockheed Martin is so grossly over budget and behind schedule that it must protect the contract at all costs. Short cuts are being taken that are not SEAL or SEI/CMM compliant. Current F-35 metrics are neither reliable nor accurate causing Lean Six Sigma implementation to be a fraud. FUTURE CHALLENGES DoD projects will only get more complicated and costly. The LOC amount continues to increase. To reduce costs, you need a bigger pot of investors. There will likely be more stakeholders increasing the complexity of requirement engineering. One example of a foreseeable future project is in Figure 10: F-35 Exports. ballistic missile defense. Both the
  • Future of DoD Metrics 12 US and NATO members will integrate with pre-existing sensors, radar installations, and weapons systems. This is due to the perceived threat from both Iran and North Korea. In 2009, North Korea reportedly tested a nuclear device. In the near future there will two frontiers that the US will need to compete for supremacy. These are cyber and space warfare. This can be seen when the Chinese government conducted an anti-satellite missile test in 2007 and the US followed by destroying another satellite. The US is worried about a sudden attack on their satellite infrastructure. Former Secretary of Defense Donald Rumsfeld nicknamed this a “Space Pearl Harbor.” As terrorists become more sophisticated, it is predicted that they will turn to cyber terrorism. With these challenges I predict Lean Six Sigma will become more essential to US national security. In order to bring more stakeholders (i.e. allies) into these DoD projects you need transparent and objective reporting that leads to reliable project estimations. One of the biggest reasons for the DoD monopoly in high- end technology is due to economic power. As developing 3rd world countries (e.g. India, China, and Brazil) Figure 11: F-35A. evolve, wealth will be disseminated more evenly. The technology gap between DoD and other organizations will be reduced. The DoD projects that have cost overruns in the billions will push potential stakeholders to partner with other organizations. In strategic military thinking, it is not about just quality but also quantity. Increased costs means less quantity that may increase geopolitical risk. So it is essential to establish stable cost and use honest reporting. The DoD will also have to manage requirements more carefully. Defects or changes in requirements will impact cost the most. A good example of this is when increasing the size of the internal weapon bay, on the F-35, increased the weight of the STOVL variant leading to a costly redesign. Figure 11 displays the F-35A. Lean Six Sigma can play a pivotal in the development of software-intensive systems. With the integration of complex systems a few bugs can cause significant damage. Complexity is a double-edged sword. With greater capability comes greater risk from cyber attacks or electronic warfare. Lean Six Sigma provides some means of ensuring standards and quality. Bugs must be managed, tracked, and resolved before they can be exploited. When the F-22 crossed the International Date Line all navigation was loss. If this was exploited, these F-22s would have not known their own location. The system must also incorporate continuous improvement throughout the system’s lifetime. Some Lean implementation can be seen in the assembly line methods of the F-35. Manufacturing must be able to maintain production standards. Lean Six Sigma records defects from each manufacturing process. Processes that add defects, or slow production, can be examined for a suitable corrective measure. One of the reasons the US was so
  • Future of DoD Metrics 13 effective in WWII is due to its manufacturing. The US produced weapons in greater quantity than the Axis. Military production is directly linked to any country’s ability to defend itself. Lean Six Sigma gives DoD the edge in manufacturing complex weapon systems faster than anyone else. All of the US military services study Lean Six Sigma as well as the DoD for this very reason. Procedures are now needed to ensure all contractors adhere to all DoD standards and report metrics accurately. If the data is bias or under-reported then the reliability of the data is questionable. Allies will not invest in business that will put their own national security at risk. Weapon development is always under the watchful eye of the Government Accountability Office due to the high costs. In 2009 Secretary of Defense Dr. Robert M. Gates stated that he wants to cut weapon systems that have unreasonable requirements and costs. The DoD might even switch to fixed-price defense contracts. If this new paradigm takes place, the DoD contractors might find it in their best interests to honestly record data and correctly estimate cost. RECOMMENDATIONS Lean Six Sigma is essential for US national security interests. The issues of fudging or withholding data have to do with the contractors themselves. The DoD should never expect defense contractors to be honest and all should be audited on a regular basis by a third party. Military services have a habit of fielding faulty technology then relying on upgrading when a fix becomes available. Requirements analysis, traceability, and management should be the focus of future DoD projects. Defense contractors will need to accurately track more data. Defects should be traced to requirements, architecture, design, code, and business processes. Sensitivity analysis should be used within change management. Here the project manager can see what processes or features are tightly coupled. For example, the requirement of an aircraft’s radar changes to allow more functionality increasing the number of transmitters and receivers needed. This causes an increase in radar weight. Then the aircraft becomes nose heavy decreasing performance. Having complex traceability you can pull the strings in a computer model to see the cause and effect. This will allow stakeholders to understand how requirement changes impact the project in a quantifiable way. My recommendations are more oversight to ensure proper data collection, fixed-contracts, and a new system of requirement traceability. CONCLUSIONS Lean Six Sigma plays a valuable part in DoD projects and will evolve to meet future needs. DoD most incorporate more oversight and better management of requirements. Lean Six Sigma provides project measurements important to both defect resolution and streamlining processes to be more efficient. DoD defense contracts will involve more stakeholders than ever before to reduce costs. Software intensive system will be designed to integrate within a larger military alliance and involve more functionality. These Figure 12: WWII Poster.
  • Future of DoD Metrics 14 military systems will need to be manufactured on a large scale. The production will need to be efficient to cut costs and increase quality. In 2009, The DoD is now trying to cut projects that have high cost overruns and unrealistic requirements. This is a new age of military acquisitions. Now cost is becoming a bigger business driver due to the end of the Cold War. You will see Lean Six Sigma continue to evolve to meet these challenges.
  • Future of DoD Metrics 15 GLOSSARY 5th Generation Jet Fighter – Stealth fighter planes, produced from 2005 onward, that notably have a level of advanced integrated sensors, situational awareness, and extreme performance that allows dominance over pre-existing fighter planes AESA – Active electronically scanned array is radar that digitally transmits multiple radio pulses, using different frequencies, in various directions and receives a specific pulse return when the signal bounces off an object giving the location of the object within 3 dimensions Air-To-Air – Act of destroying an air object from another air object Air-To-Ground – Act of destroying a ground object from the air AN/APG-81 – F-35 Joint Strike Fighter’s AESA radar that can simultaneously destroy air and ground targets Battlespace – 3-dimensional area of military operations that includes air, sea, land, and space CAT-bird – Cooperative avionics test bed is a Boeing 737-300 aircraft used to test F-35 software separate from the F-35 physical architecture CMM – Capability Maturity Model is an assessment development, by Carnegie Mellon University, about how mature an organization’s continuous soft development methodology is for government contract work Cold War – State of military tensions between the Soviet Union and the United States of America from1945 to1991. COTS – Commercial-off-the-self-software is commercial software used to lower cost Counter Intelligence – Prevent other organizations from gathering correct intelligence Cummins – Global leader in engine manufacturing and related technologies CV – Carrier version of aircraft DARPA – Defense Advanced Research Projects Agency is responsible for developing new technology for the US military DAS – Digital Aperture System is a set of advanced infrared cameras imbedded in a vehicle that searches, tracks, and identifies other objects DIA – Defense Intelligence Agency produces and manages foreign intelligence for military operations DoD (DOD) – Department of Defense is tasked with defending the United States of America and its interests Earned Value Management System – Project management system that compares the actual cost, earned value, and planned value to identify early warning signs Electronic Warfare – Use electromagnetic spectrum against an enemy (e.g. jamming, using energy to burn microchips, and planting false information into other sensors) EOTS – Electro-optical targeting system is a forward looking infrared, laser designator, and laser rangefinder with the primary purpose of targeting both air and ground vehicles Espionage – Obtain confidential information without permission F-15E – F-15E Strike Eagle is an US air superiority fighter, introduced to the Air Force in 1986, which is considered to be the best aircraft of all time and is currently being replaced by the F-22A Raptor and F-35 Lightning II
  • Future of DoD Metrics 16 F-22A – F-22A Raptor is the new US air superiority 5th generation stealth fighter introduced to the Air Force in 2005 F-35 – F-35 Lightning II is the new 5th generation stealth strike fighter that will be introduced in 2012 to all services and some allies GAO Government Accountability Office audits federal projects G-force – Act of acceleration on an object and is often used to determine how much stress a human body is experiencing during acceleration HMD – Helmet mounted display allows the pilot to be given information from his/hers helmet ITAR – International Traffic in Arms Regulations JPO – Joint Strike Fighter Program Office JSF – Joint Strike Fighter Kenworth – Manufacture of medium to heavy trucks LOC – Lines of code MDA – Missile Defense Agency tasked with protecting the US against ballistic missiles National War College – US college, funded by the Department of Defense, which teaches military management, strategy, and diplomacy Net-Centric Warfare – Warfare strategy that networks military assets and operations together creating competitive edge through situational awareness and concentration of force NGA – National Geospatial-Intelligence Agency specializes in mapping of foreign territory NSA – US National Security Agency specializing in cryptologic intelligence PDA – Personal Digital Assistant is a hand held computer PFPA – Pentagon Force Protection Agency protects and polices the Pentagon Prime Contractor – Contract owner QDR – Quadrennial defense review done by the Department of Defense to analyze risks to US national security and interests Radar – System that uses electromagnetic waves to track and identify objects. Active radar is when the radar actively transmits signals and passive radar is when the radar only listens to locate other active radars Requirement Analysis – Tasks that determine the procedures on how to add or alter project goals SAR – Synthetic aperture radar processes several images that when combined creates one detailed ground image SDP – Software development plans SEAL – Safety evidence assurance level is a category of required evidence ensuring a specific type of system is safe Sensitivity Analysis – Study how variations in data affect correlations Six Sigma – Quality management, through the use of statistical analysis, to identify and remove 3.4 defects per million opportunities Software Development Life Cycle (SDLC) – Systematic process of software development and implementation Software Engineering – Scientific methodology of software development, operation, and maintenance
  • Future of DoD Metrics 17 Software Engineering Institute (SEI) – World renowned federally funded software research and development center at Carnegie Mellon University Software-Intensive Systems – Any system that relies heavily on other systems or sensors SPAR – Lockheed Martin’s software problem anomaly reports that tracks defects and audited by the Department of Defense SPE – Lockheed Martin’s software product evaluation data Stealth Fighter – Military aircraft that incorporates significant low observability to hide from radar, infrared, visual, acoustic, and radio frequencies STM – Lockheed Martin’s software management team STOVL – Aircraft that can take off on short runways and land vertically Strike Fighter – Military aircraft that has the primary purpose of attacking surface targets Subcontractor – Individual or business that signs a contract to perform part of another pre-existing contract Toyota – Largest multination car manufacturer by sales US – United States of America WWII – World War II was a global war that started in 1939 and ended in 1945 when the Allies defeated the Axis.
  • Future of DoD Metrics 18 CITED REFERENCES 1. U.S. Department of Defense’s Office of the Deputy Chief Management Officer [Internet]. Washington (DC): Strategic Management Plan; [cited 2009 Nov 11]. Available from: 2. U.S. Department of Defense’s Office of the Deputy Chief Management Officer [Internet]. Washington (DC): Strategic Management Plan; [cited 2009 Nov 11]. Available from: 3. United States Government Accountability Office (US). Increased Focus on Requirements and Oversight Needed to Improve DoD’s Acquisition Environment and Weapon System Quality. Report to Congressional Committees. Washington (DC): United States Government Accountability Office (US); 2008 Feb [cited 2009 Nov 13]; GAO-08-294. p. 1. Available from: 4. United States of America ex rel. Sylvester Davis v. Lockheed Martin Corporation., Case: 1:06-cv-0013-RLF-GWC (U.S. District of Virgin Islands St. Croix Division filed 05/26/2009) [cited 2009 Nov 14] Available from: Martin-Lockheed NON-CITED REFERENCES 5. Lockheed Martin [Internet]. Bethesda (MD): F-35 Lightning II Future is Flying; [cited 2009 Mar 17]. Available from: roc.pdf. 6. Information Technology for European Advancement. Software Intensive Systems in the Future. [report on the Internet]. Internet: IDATE; 2005 [cited 2009 Mar 20]. Available from: 7. Baxter G. Lightning II F-35 Lessons Learned. In: International Support Equipment Conference; 2007 [cited 2009 Mar 17]. p. 1-29. Available from: ISECLessonsLearned_05_22_2007.ppt. 8. United States Government Accountability Office (US). Defense Acquisitions Assessment of Selected Major Weapon Programs Support. Report to Congressional Committees. Washington (DC): United States Government Accountability Office (US); 2005 Mar [cited 2009 mar 20]; GAO-05-301. p. 79-80. Available from:
  • Future of DoD Metrics 19 9. Strategic Thought Group [Internet]. Washington (DC): Risk Management Case Study; 2007 [cited 2009 Mar 17]. Available from: Study-15363.pdf. 10. Introduction to CMMI Acquisition Module (CMMI-AM) Module 1 [Internet]. Software Engineering Institute: Carnegie Mellon; 2005 [cited 2009 Mar 17]. Available from: 11. AllBuisness [Internet]. San Francisco (CA): Production targets: F-35 factory: one aircraft per day by 2016; [cited 2009 Nov 18]. Available from: aircraft/11421207-1.html 12. Didier J. DOD Risk Analysis [Internet]. Chicago (IL): DePaul University; 2009. [cited 2009 Nov 18] Available from:
  • Future of DoD Metrics 20 DIAGRAMS