Successfully reported this slideshow.
Your SlideShare is downloading. ×

From Principles to Strategies for Systems Engineering

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
Heliotropic Abundance
Heliotropic Abundance
Loading in …3
×

Check these out next

1 of 268 Ad

From Principles to Strategies for Systems Engineering

Download to read offline

From Principles to Strategies How to apply Principles, Practices, and Processes of Systems Engineering to solve complex technical, operational,
and organizational problems

From Principles to Strategies How to apply Principles, Practices, and Processes of Systems Engineering to solve complex technical, operational,
and organizational problems

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to From Principles to Strategies for Systems Engineering (20)

Advertisement

More from Glen Alleman (20)

Recently uploaded (20)

Advertisement

From Principles to Strategies for Systems Engineering

  1. 1. Systems Engineering From Principles to Strategies How to apply Principles, Practices, and Processes of Systems Engineering to solve complex technical, operational, and organizational problems 1
  2. 2. Course Contents § Lesson 0 – Course Introduction § Lesson 1 – Overview of SE Discipline § Lesson 2 – Principles of Systems Engineering § Lesson 3 – Practices of Systems Engineering § Lesson 4 – Processes of Systems Engineering § Lesson 5 – Applying SE to Notional Project Hands On Project Used To Illustrate these lessons throughout the course 2
  3. 3. Systems Engineering in One Picture 3 † National Airspace Systems Engineering Manual, Federal Aviation Administration, ATO Operations Planning, Oct 11, 2006
  4. 4. Exit Criteria for this Course § Provide the skills and knowledge needed to apply Systems Engineering in any program domain § Develop understanding of the Principles, Practices, and Processes of Systems Engineering needed to increase the probability of program success (PoPS)† – Capabilities Based Planning – Measures of Effectiveness and Performance of the program outcomes – Provide the mechanisms to convert user needs into user satisfaction 4 † Probability of Program Success (PoPS) is a framework implemented across DOD services. “Implementation Guidance and Methodology for Naval Probability of Program Success (PoPS), Office of Assistant Secretary Research, Development, and Acquisition, 1000 Navy Pentagon, Washington DC, Oct 06, 2008
  5. 5. Start With The End In Mind § Systems Engineering provides the guiding principles for the analytic foundation of any development program through – Technical Frameworks – Requirements Management – Integration Management – Risk Identification and Mitigation – Evaluation, Verification, and Validation of all work products 5
  6. 6. We’ve Met the Enemy and He is Us† § Unrealistic performance expectations § Unrealistic baseline estimates for cost and schedule § Immature technologies or excessive manufacturing or integration risk § Unanticipated design, engineering, manufacturing, or technology integration § Issues arising during program performance § Changes in procurement quantities § Inadequate program funding or funding instability § Poor performance by customer or contractor personnel responsible for program management 6 † WSARA (2009) lists eight root causes in Decisions Made During Program Execution as a Root Cause of Nunn-McCurdy Breaches , The Evidence from Root Cause Analyses done by RAND and IDA for PARCA May 16-17 David L. McNicol
  7. 7. Lesson 0 Systems Engineering is a disciplined approach to solving complex technical, organizational, and operational problems, using Systems Thinking to consider the whole rather than just a collection of the parts. 7
  8. 8. Lesson 0 § Why are we here? § What is Systems Engineering? § Systems Engineering Framework 8 Lesson 0
  9. 9. Why Are We Here These 3 Days? § All the worlds system, we need to learn how to manage the processes of developing, then assembling. all these moving parts. § Systems Engineering is the means to delivering the final system made up of these parts. § This course will provide you with the Principles, Practices, and Processes needed to increase the probability of program success using Systems Engineering. 9 Lesson 0
  10. 10. All the worlds’ a system. All the parts are separate. All the parts are connected. 10 Lesson 0
  11. 11. Systems Thinking § Before moving the Systems Engineering let’s talk about Systems Thinking § Russell Ackhoff† tells us systems are defined by – Parts – Relationships – Purpose § Systems Thinking looks at – Relationships – rather than unrelated objects – Connectedness – rather than structure – The whole – rather than just its parts – Patterns – rather than contents 11 † Systems Thinking for Curious Managers, R. Ackoff, with H. Addison and A. Carey, Triarchy Press 2010 Lesson 0
  12. 12. Systems Thinking § Thinking systematically … requires several shifts in perception, which lead in turn to different ways to teach, and different ways to organize society† § Our perception needs to move away from … – Take the thing or event to be understood apart – Explain the behavior or properties of the parts taken separately – Aggregate the explanations of the parts into an understanding of the whole, the thing to be explained. 12 † Redesigning Society, Ackhoff and Rovin, Stanford Business School Books, 2003 Lesson 0
  13. 13. Moving to Synthetic Thinking† § Managers should never accept the output of a technologically-based system unless they understand exactly what the system does and why. § Systems must be understood in the context of what they can do and the world in which they will do it. § It is not enough to see the system as a sum of the operations of the component parts. § It must be seen as a functioning whole. § This is the System Thinking Point of View. 13 † A Primer for Model-Based Systems Engineering, 2nd Edition, David Long and Zane Scott, Vitech, 2011 Lesson 0
  14. 14. Systems Engineering is…† “an interdisciplinary approach to translating users’ needs into the definition of a system, its architecture and design through an iterative process that results in an effective operational system. Systems engineering applies over the entire life cycle, from concept development to final disposal.” 14 † Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Acquisition (2008) Lesson 0
  15. 15. We Will To Learn To Identify All The Parts And Assemble Them Into A Functioning System And Learn How All The Elements Of Systems Engineering Will Aide Us In That Process 15 Lesson 0
  16. 16. World of Engineered Systems† 16 Systems Engineering Systems Architecture Requirements Definitions Stakeholder Analysis Trade Space Exploration Design Definition and Optimization Human Factors Systems Integration Interface Management Verification and Validation Commissioning and Operations Lifecycle Management † Derived from MIT Open Courseware, readings for Systems Engineering Lesson 0
  17. 17. It All Started Here The term Systems Engineering dates to Bell Telephone Laboratories in the early 1940s. The first attempt to teach Systems Engineering as we know it today came in 1950 at MIT by Mr. Gilman, Director of Systems Engineering at Bell. Systems engineering was defined as work functions with five phases: 1. System studies or program planning 2. Exploratory planning, including problem definition, selecting objectives, systems synthesis, systems analysis, selecting best system, and communicating the results. 3. Development planning, repeating phase 2 in more detail. 4. Studies during development, including development of parts of the system and integration and testing of these parts; 5. Current engineering, taking place while the system is operational and being refined. RAND Corporation created systems analysis, an important part of Systems Engineering. The Department of Defense entered the world of Systems Engineering in the late 1940s with the development of missiles and missile-defense systems. Paul Fitts† codified Systems Engineering by addressing the allocation of the systems functions to the physical elements of the system in the late 1940s and early 1950s. † Paul Fitts (ed) (1951) Human engineering for an effective air navigation and traffic control system. National Research Council, Washington, DC 17 Lesson 0
  18. 18. And Now, Systems Are Everywhere 18 Lesson 0
  19. 19. Complicated, Complex, and Complexity § Complicated – consisting of many interacting parts or elements § Complex – characterized as many parts that interact with in multiple ways with possibly unknown outcomes § Complexity – the state of being intricate or complicated § Irreducible Complexity - characteristic of certain complex systems in which all components needed to function. 19 Lesson 0
  20. 20. Systems Engineering is about Managing Complexity† 20 Product Number of Parts Number of Levels Screwdriver 3 1 Roller Blades 30 2 Inkjet Printer 300 3 Copy Machine 2,000 4 Automobile 10,000 5 Airliner 100,000 6 † Modified from, Ulrich, K.T., Eppinger S.D. , Product Design and Development, Second Edition, McGraw Hill, 2000, Exhibit 1-3 Lesson 0
  21. 21. 21
  22. 22. The Structure of This Course § Lessons, 1 through 4, – Overview, – Principles, – Practices, and – Processes of Systems Engineering. § Learning assessments of these concepts are at the end of each Lesson. § A notional project is used for classroom experience to apply the Principles, Practices, and Processes. 22 Lesson 0
  23. 23. We Will Focus On … § INCOSE and 15288 Processes § The system life cycle management § Project Management Processes § Using our notional project we will: – Develop Measures of Effectiveness (MOE), Performance (MOP), and Technical Performance (TPM) – Trace the Principles, Practices, and Processes to our notional program 23 Lesson 0
  24. 24. Overview Of Our Course Lesson 2 Principles Lesson 3 Practices Lesson 4 Processes SE Lifecycle(s) Vee Paradigm SEMP UAV Project ConOps SOW WBS MOE MOP TPM 24 Lesson 5 Notional Project Lesson 0
  25. 25. When We Say Systems Engineering What Do We Mean?† § System – A combination of interacting elements organized to achieve one or more stated purposes. § Systems of Interest – The system whose life cycle is under consideration in the context of INCOSE and 15288 Guidance. § System Element – A member of a set of elements that constitutes a system. – A system element is a discrete part of a system that can be implemented to fulfill specified requirements. § Enabling System – A system that complements a system-of-interest during its life cycle stages but does not necessarily contributed directly to is function during operation. – When a system-of-interest enters the production stage, an enabling production system is required. 25 † Extracted from ISO/IEC 15288 Lesson 0
  26. 26. Through Classroom Examples, We’ll Learn Systems Engineering for … § Capabilities Elicitation § Requirements Development § Product Development – Hardware – Software – Operations, maintenance, and support § Integration And Test § Operational Verification and Validation § Deployment, Support, and Maintenance 26 Lesson 0
  27. 27. With This Knowledge We Will Confirm the Needed Capabilities can be Delivered as Planned† § System provides balanced and optimized products that meet the customers needed capabilities. § Effective requirements analysis is applied in the delivery of these capabilities . § Consistent and rigorous application of Systems Engineering management standards is applied throughout the program’s lifecycle. § Effective planning to test these capabilities is accomplished. § Effective major technical program reviews to evaluate the emerging capabilities are conducted. § Continuous risk assessments and management to assure capabilities are delivered as planned is implemented. § Reliable cost estimates and policies for the delivery of the capabilities are developed. § Disciplined application of configuration management are demonstrated. § System boundaries are well-defined. † Global Hawk Systems Engineering Case Study, Air Force Center for Systems Engineering, Wright Patterson AFB, 2010. 27 Lesson 0
  28. 28. 28 Lesson 0
  29. 29. CAPABILITIES BASED PLANNING† Before proceeding to the next steps in Systems Engineering, let’s pause and discuss an issue in all modern program developments. Many programs provide a multitude of technical and operational features and functions. We’ve all experienced this. Software tools, automobiles with more features than we can remember, complex systems like aircraft with features so complex the pilots have trouble remembering how to operate then (one cause of the Asiana Air crash in SFO is attributed to the multiple features in decent control system that created confusion). One improved approach to engineering a system is to determine what capability is needed to accomplish the mission or provide a solution to the business problem. 29 † Guide to Capabilities Based Planning, The Technical Cooperation Program, Joint Systems and Analysis Group, Technical Panel 3. Lesson 0
  30. 30. Capability Versus Requirement § Capabilities describe three characteristics needed before proceeding to requirements elicitation – Possible scope – delimits the boundaries of the problem and the possible solutions – Possible forms – specific characteristic of a process not related to its application – Possible solutions – candidate solutions that have already been applied to solve similar problems that should be investigated to determine if they should be part of the this solution Lesson 0 Provide safe transport to the public is a capability. No single failure in he breaking system shall endanger life is a requirement. 30
  31. 31. Capabilities-Based Planning† § Capabilities-Based Planning is planning, under uncertainty, to provide capabilities suitable for a wide range of business challenges and circumstances, while working within an economic framework § Capabilities-Based Planning emphasizes flexibility, adaptiveness and robust capabilities, implying a modular building-block approach to Enterprise Services § When transformation takes place it is because new modules have come into use 31 Lesson 0 † Analysis to Support Capabilities-Based Planning, Tom Allen, Institute for Defense Analyses, Capabilities-Based Planning Workshop, 19-21 October 2004
  32. 32. A Simple Example Our system of provisioning a new employee 32 Human Resources New Employee Ready to Work Insurance Orientation Laptop Account Setup Charge account setup Information Technology Finance Buying authority available Supply Chain Management We Need the Capability To Provide Buying Authority within 2 working days of new employee hire Lesson 0
  33. 33. Capabilities State the Intent of the Commander 33 Action Outcome Implement Strategy Ensure Capabilities Prioritize Problems And Solutions Identify Redundancies Deliver Solutions † Photo Eustachy Kossakowski Lesson 0 In preparing for battle I have always found that plans are useless, but planning is indispensible
  34. 34. The Role SE in Identify Needed Capabilities and Managing Their Delivery What Should We Do? Where Are We Now? Identify Capability Mismatches Operational Concepts Capability Goals Scenarios Priorities Customer Needs Investment Balance Solution Deployment Options Capability Assessment Mission Priorities Resource Constraints Capability Partitions Current and Planned Capabilities Affordable Capabilities Plan Abstracted from: “Capabilities‒Based Planning – How It Is Intended To Work And Challenges To Its Successful Implementation,” Col. Stephen K. Walker, United States Army, U. S. Army War College, March 2005 Future Environment Lesson 0 34
  35. 35. A Reminder Before We Start† § Systems Engineering Is … – A logical sequence of activities and decisions that transforms an operational need into a description of system performance parameters and a preferred system configuration. (MIL-STD-499A, Engineering Management) – An interdisciplinary approach that encompasses the entire technical effort, and evolves into and verifies an integrated and life cycle balanced set of system people, products, and process solutions that satisfies a customer need. (EIA Standard IS-632, Systems Engineering) – An interdisciplinary, collaborative approach that derives, evolves, and verifies a life cycle balanced system solution that satisfies customer expectations and meets public acceptability (IEEE P1220 Standard for Application and Management of Systems Engineering Process) 35 † These definitions have been replaced by INCOSE and 15288, but serve as reminders of the depth and breadth of Systems Engineering as the foundation of program success. Lesson 0
  36. 36. One More Reminder § Systems Engineering is about the system aspects of program outcomes. § Systems Engineering does not build the products of the program outcomes, it enables the right products to be built right, in the right way, at the right time, to deliver the right value to the customer. 36 Lesson 0
  37. 37. Oops, One Final Reminder A system is a collection of elements, that together, produce result not obtained by the elements alone. These elements can include people, hardware, software facilities, policies, and documents – all interacting to produce on outcome. 37 System Engineering is a product oriented discipline whose responsibility is to create and execute the interdisciplinary processes that ensure the stakeholder needs are satisfied. Lesson 0
  38. 38. Learning Objectives for the Systems Engineering Course § Understand the most important Systems Engineering standards and best practices and how these guide the principles, practices, and processes of Systems Engineering § Understand the key steps in system engineering process starting with stakeholder analysis and ending by transitioning the system to operations § Understand the role of the human aspects of Systems Engineering § Apply these understandings to a notional project to Connect the Dots for actual use after the course 38 Lesson 0
  39. 39. Lesson 0 39 Science determines what IS Component engineering determines what CAN BE Systems Engineering determines what SHOULD BE† † Special Feature, INCOSE Insight, Vol 5, Issue 1, April 2002
  40. 40. Lesson 1 Introduction to the Discipline of Systems Engineering Using ISO/IEC 15288 and INCOSE Systems Engineering Handbook 3.2 40
  41. 41. How To Think Like A Systems Engineer § Systems Thinking says its name – A framework for solving problems where the component part of an entity is best understood in the context of its relationship with other components of the entity. § These are fancy words for – Understanding any Part, Problem, or Solution through its relationship with the Whole 41 Lesson 1
  42. 42. As a Discipline, Systems Engineering …† § Matches the product to the marketplace § Defines the components so the designers can be design and built them § Determines most of the design choices affecting system cost and performance § Ensures that the components will integrate successfully and perform together as required § Provides specifications free of errors, since errors are very expensive to correct in the latter stages of design and production. 42 Lesson 1 † Engineering Complex Systems with Models and Objects, David W. Oliver, Timothy P. Kelliher and James G. Keegan, Jr., McGraw-Hill
  43. 43. First Principle of Systems Engineering “All Systems Engineering models and processes are organized around the concept of a life cycle. The detailed views, implementations, and terminology that articulate the life cycle differ across organizations and customers, they share fundamental elements guided by Systems Engineering standards.” † 43 Lesson 1 † From MITRE SE Life-Cycle Building Blocks
  44. 44. Some System Engineering Standards § ANSI/GEIA EIA-632, Processes for Engineering a System, 01 Sept 2003 § ISO/IEC 15288 – System Lifecycle Processes § EIA/IS 731.1, Systems Engineering Capability Model, Electronic Industries Alliance (Interim Standard), 01 Aug 2002 § IEEE 1220-2005, IEEE Standard for Application and Management of the Systems Engineering Process, Institute of Electrical and Electronics Engineers, 09 Sept 2005 § ISO/IEC 15504: 2004 - Information Technology - Process Assessment § INCOSE Systems Engineering Handbook, Systems Engineering Handbook A Guide For System Life Cycle Processes And Activities Incose-TP-2003-002-03.2 January 2010 44 Lesson 1
  45. 45. INCOSE SE Handbook† § Generic life cycle § Technical Processes § Project Processes § Agreement process § Organizational processes § Tailoring processes § Specialty Engineering 45 Lesson 1 † INCOSE Systems Engineering Handbook can be found at www.incose.org
  46. 46. ISO/IEC 15288 Standard § 25 processes, 123 outcomes, derived from 403 activities and 6 examples covering life cycle of human made systems § System is composed of elements, where they are implemented and integrated into the system § Elements can themselves be considered a system. 46 Lesson 1
  47. 47. ISO/IEC 15288 Framework Agreement Organization Project Technical Concept Develop Produce Use Retire 47 Support Lesson 1
  48. 48. ISO 15288 Systems Engineering Processes Enterprise Process § Enterprise Process Management § Investment Management Processes § Systems Lifecycle Processes § Resource Management Processes Agreement Process § Acquisition § Supply Project Processes § Planning § Assessment § Control § Decision Management § Risk Management § Configuration Management § Quality Management Technical Processes § Stakeholder needs definition § Requirements analysis § Architectural design § Implementation § Verification § Integration § Transition § Validation § Operation § Maintenance § Disposal 48 Lesson 1
  49. 49. Defense SE and Commercial SE 49 Concept Stage Development Stage System Definition Preliminary Design Detailed Design FAIT Production Stage Utilization Support Stage Customer Support Retire ISO 15288 IEEE 1220 Lesson 1
  50. 50. Systems Engineering is Applicable in a Wide Variety of Domains § Aerospace § Telecommunications § Transportation Systems § Military Systems § Ship Building § Finance and Administrative Systems § Information Technology Systems 50 Source: ISO/IEC JTCI/SC7/WG7 Source: ISO/IEC JTCI/SC7/WG7 presentation on ISO/IEC 15288. Lesson 1
  51. 51. Our Lingua Franca for Engineering Systems using INCOSE and 15288 § Every system has a lifecycle, viewed as stages § Stages are separated by decision gates § Stages may overlap or be concurrent § Each stage is accomplished by executing processes within the stage § Any process may be applied to any stage 51 Lesson 1
  52. 52. What is Systems Engineering? § Systems Engineering says its name. It … … is an interdisciplinary field of engineering focused on the design and management of complex project throughput their life cycle. It addresses the disciplines of reliability, logistics, coordination of teams, requirements management, evaluation metrics, optimization methods, risk management – the System. … integrates all the disciplines and specialty groups into a team effort forming a structured development process that proceeds from concept to production to operation. Systems Engineering considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs. 52 Lesson 1
  53. 53. The Primary Structure of Systems Engineering Two Distinct Systems Engineering Activities† From Partitioning è To Integrating User Needs è User Satisfaction User Requirements è User Acceptance Test System Requirements è System Acceptance Test System Design è Integration Test System Development è Subsystem Test 53 Lesson 1 † This is the foundation of the Systems Engineering Vee
  54. 54. Typical Life Cycle Management 54 Cycle Activities Exploratory Identify enabling technologies Concept Analyze needs, identify concepts, and develop solutions Development Engineer system to deliver needed capabilities Production Build the system to deliver needed capabilities Utilization Operate the system for the benefit of the user Support Maintain and support the system Retire Retire, archive, or dispose of the system Lesson 1 1 2 3 4 5 6 7
  55. 55. Exploratory § Create reference architecture § Identify enabling technologies § Generate early cost and schedule projections § Clearly identify needed customer capabilities § Assess technology readiness 55 1 Lesson 1 Life Cycle Management
  56. 56. Concept § Refine and broaden any studies § Focus on requirements driven approaches rather than product driven § Identify, clarify, and document stakeholder requirements § Build in-depth studies to evaluate multiple candidates § Provide sustained justification for system concept § Prototypes used to verify feasibility of concepts § Expand risk and opportunity evaluation for affordability, environmental impacts, failure modes, hazard analysis, technical obsolescence, and system disposal 56 2 Lesson 1 Life Cycle Management
  57. 57. Development § Detailed planning, development, and IV&V § Incremental and iterative processes apply here § Full freedom to apply development model – Waterfall – Plan driven – Agile development 57 3 Lesson 1 Life Cycle Management
  58. 58. Production § Produce system-of-interest, manufactured product, or delivered service § Modify product to resolve production issues, reduce costs, enhance product capabilities § Systems engineering assessment of any changes during production. 58 4 Lesson 1 Life Cycle Management
  59. 59. Utilization § Operate system-of-interest § Plan product modification through out operation of system § Upgrades and enhancements to capabilities 59 5 Lesson 1 Life Cycle Management
  60. 60. Support § Provide services that enable continued operation. § Propose modifications to resolve: – Supportability problems – Reduce operational costs – Extend life of the system § Assess changes to avoid loss of system capabilities while under operation. 60 6 Lesson 1 Life Cycle Management
  61. 61. Retirement § System of interest and related services removed from operation § Ensure retirement requirements satisfied § Retirement planning done during Concept process 61 7 Lesson 1 Life Cycle Management
  62. 62. Consensus of INCOSE Fellows for Core Framework for Systems Engineering† State the Problem Investigate Alternatives Model the System Integrate Developed Components Launch Resulting System Access Performance Re-Evaluate 1 2 3 4 5 † A Consensus of INCOSE Fellows on What is Systems Engineering? The Systems Engineering Process from A. T. Bahill and B. Gissing, Re-evaluating Systems Engineering concepts using systems thinking, IEEE Transaction on Systems, Man and Cybernetics, Part C: Applications and Reviews, 28 (4), 516-527, 1998. This Is An Iterative And Parallel Process 6 7 62 Lesson 1
  63. 63. State the Problem† § Describe the top level functions in terms of capabilities the system must provide for success § State the problem to be fixed (deficiency that will be ameliorated) § State mandatory needed for acceptability § State the Measures of Effectiveness and Measures of Performance needed for success 1 63 Lesson 1 Core Framework † Derived from Bahill, A. T. and Dean, F. F., Discovering system requirements, Chapter 4 in Handbook of Systems Engineering and Management, A. P. Sage and W. B. Rouse
  64. 64. Investigate Alternatives § Create an evaluate alternatives based on technical and operational performance, cost, and schedule § Use multi-criteria decision making processes to assess alternatives § Establish model elements, their coupling and cohesion, to be tested next 2 64 Lesson 1 Core Framework
  65. 65. Model the System § Models can be used to test alternatives for most system designs. § Models represent the essential characteristics of the system under development § The preferred model can be expanded and used throughout the program lifecycle § Many models available – Physical analogs – plywood and lead weights for spacecraft – Mathematical models – 3D CATIA† dynamic models – Block diagrams, state machines functional flow models, computer simulations – can model dynamic behaviors 3 65 Lesson 1 Core Framework
  66. 66. Integrate Developed Components § The system, the business processes implemented by the system, and the people participating in that business process must all be integrated. § Define the subsystems on natural boundaries – This is a hard problem, but start with coupling and cohesion considerations – Use an Interface Control Document paradigm to manage connections to the subsystems § Integration between co-evolving subsystems is high risk and must be explicitly managed 4 66 Lesson 1 Core Framework
  67. 67. Launch Resulting System § Deploy the system, run it, and produce outputs. – More complex than it sounds of course – So need a Plan for the Plan to deploy, run and produce with minimum risk and maximum value § SE products include: – Mission capabilities description – Technical and operational requirements – Allocation of functions to physical components – Measures of Effectiveness, Performance, and Technical attributes used to assess capabilities will be provided 5 67 Lesson 1 Core Framework
  68. 68. Assess System Performance § Each level of the system, from abstract to tangible, needs a measures of its outcome 6 System Measure Capability Measures of Effectiveness (MOE) Requirement Measures of Performance (MOP) Product Technical Performance Measures (TPM) Mission Attribute Key Performance Parameters (KPP) 68 Lesson 1 Core Framework
  69. 69. Measures of Effectiveness (MOE) 69 Measures of Effectiveness … § Are stated in units meaningful to the buyer, § Focus on capabilities independent of any technical implementation, § Are connected to the mission success. The operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions. Measures of Effectiveness Belong to the End User Lesson 1 6 Core Framework
  70. 70. Measures of Performance (MOP) 70 Measures of Performance are … § Attributes that assure the system has the capability to perform, § Assessment of the system to assure it meets design requirements to satisfy the MoE. Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. Measures of Performance belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C Lesson 1 6 Core Framework
  71. 71. Key Performance Parameters (KPP) 71 Key Performance Parameters … § Have a threshold or objective value, § Characterize the major drivers of performance, § Are considered Critical to Customer (CTC). Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program The acquirer defines the KPPs during the operational concept development – KPPs say what DONE looks like Lesson 1 6 Core Framework
  72. 72. Technical Performance Measures (TPM) 72 Technical Performance Measures … § Assess design progress, § Define compliance to performance requirements, § Identify technical risk, § Are limited to critical thresholds, § Include projected performance. Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal Lesson 1 6 Core Framework
  73. 73. Dependencies Between These Measures 73 MoE KPP MoP TPM Mission Capabilities Acquirer Defines the Needs and Capabilities in terms of Operational Scenarios Supplier Defines Physical Solutions that meet the needs of the Stakeholders Operational measures of success related to the achievement of the mission or operational objective being evaluated. Measures that characterize physical or functional attributes relating to the system operation. Measures used to assess design progress, compliance to performance requirements, and technical risks. Lesson 1 Core Framework 6
  74. 74. Re-evaluate Resulting Outputs § The most important function of the systems Engineering framework is the re-evaluation of each phases outcomes at each phase. § This is continuous feedback § Optimism is the disease of all technical development, feedback is the cure† 7 74 Lesson 1 Core Framework † Kent Beck, software engineer and creator of eXtreme Programming and Test Driver Development Customer Need State the Problem Investigate Alternatives Model the System Integrate Launch the System Assess Performance Outputs Re-Eval Re-Eval Re-Eval Re-Eval Re-Eval Re-Eval
  75. 75. What Roles Do Systems Engineers Play In This Framework?† Enterprise Perspective Engineering Life Cycle Paradigm Engineering Planning and Management Technical Specialties Collaboration 75 Lesson 1 1 2 3 4 5 † Adapted from MITRE Systems Engineering (SE) Competency Model, September 1, 2007, Version 1.13E Sework Program
  76. 76. System Engineering Roles† 76 Lesson 1 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  77. 77. Enterprise Perspective† § Comprehensive viewpoint – A broad understanding of the system and the context in which it “Lives” – Discover new approaches and ideas to modeling complex systems – Provides long term strategies to achieve mission objectives and exploit opportunities 77 Lesson 1 1 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  78. 78. Enterprise Perspective† § Innovative approaches – Partial solution provided to ambiguous problems – Frame the “essence” of the problem – Provides scalable and adaptable solutions to complex system solutions § Enable stakeholder relations – Leverage engineering resources – Communicate independent, objective, direct information – Facilitate decision-making 78 Lesson 1 1 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  79. 79. Systems Engineering Life Cycle† § Concept definition – Establish agreement on mission need and Concept or Operations – Develop high-level conceptual design § Requirements Engineering – Integrate mission and operational needs into system requirements – Facilitate agreement on changes to requirements 79 Lesson 1 2 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  80. 80. Systems Life Cycle Engineering† § Architecture – Identify alternative architecture solutions § System design and development – Establish agreement on design review and milestone decision approaches § Systems integration – Provide integration strategies to meet mission need, integration and interoperability challenges 80 Lesson 1 2 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  81. 81. Systems Engineering Life Cycle† § Test and Evaluation – Guide test and evaluation strategies of an effective and interoperable system – Influence stakeholders on mitigation strategy and system acceptance § System Implementation, OM&T – Develop agreement on transitional approach and system deployment – Influence approaches to system modification and technology insertion 81 Lesson 1 2 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  82. 82. Planning and Management† § Transformational Planning – Develop strategy for transforming stakeholder organization, structure, processes, system interactions with other organizations – Collaborate and build consensus with stakeholders for transforming organization 82 Lesson 1 3 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  83. 83. Planning and Management† § Government Acquisition Support – Collaborate with stakeholders to establish acquisition program and program office – Work with stakeholder to select the Systems Engineering life cycle approach – Recommends best value offeror to stakeholder § Contractor Evaluation – Influence stakeholder decision during contractor evaluations and milestone reviews – Recommend changes based on contractor performance 83 Lesson 1 3 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  84. 84. Planning and Management† § Risk Management – Influence risk management approach – Influence risk mitigation strategies and program direction § Configuration Management – Influence configuration management approach § Integrated Logistics Support – Recommend and implement integrated life cycle logistics support strategy – Guide approval and implementation of ILS alternatives 84 Lesson 1 3 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  85. 85. Planning and Management† § Quality assurance and Measurement – Guide establishment and direction of quality assurance program in the systems acquisition – Influence resolution of corrective actions to ensure adherence to processes § Continuous Process Improvement – Influence approach for implementing and improving Systems Engineering processes – Influence stakeholder organizations to finalize, implement, and continuously improve Systems Engineering processes 85 Lesson 1 3 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  86. 86. Technical Engineering Specialties† § Cost / Benefit Analysis – Provide direction to analyst for scope, key parameters, products, and tradeoffs of the analysis – Provide strategic, programmatic and enterprise- wide perspective of cost/benefit analysis 86 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  87. 87. Technical Engineering Specialties† § Human Centered Engineering – Recommend design and trade-off decisions during the early acquisition phases, based on a human- centered engineering approach. – Collaborates with the HCE and the sponsor/customer to resolve HCE issues. 87 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  88. 88. Technical Engineering Specialties† § Modeling And Simulation – Recommend M&S scope, approaches, and changes to operational capabilities – Facilitates cooperative modeling arrangements. 88 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  89. 89. Technical Engineering Specialties† § Security Engineering – Identify security engineering approaches and constraints – Plan for certification and accreditation, – Determine security related trade-offs. 89 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  90. 90. Technical Engineering Specialties† § Reliability, Maintainability, And Availability – Collaborates with RMA specialist and the sponsor/customer to identify approaches, interpret model results – Determine design changes and prioritized corrective actions. 90 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  91. 91. Technical Engineering Specialties† § Safety Engineering – Recommends and gains consensus on safety engineering approaches, design trade-offs, and solutions 91 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  92. 92. Technical Engineering Specialties† § Network And Communications Engineering – Interacts with the specialist to facilitate task completion and makes recommendations to the sponsor/customer on network and communication issues. 92 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  93. 93. Technical Engineering Specialties† § Software and Information Engineering – Facilitates interaction among the sponsor/customer, end-users, and software engineers to determine system expectations, problems, and potential solutions – Gains consensus with the sponsor/customer on an information-centric view of the enterprise – Communicates risks and mitigation strategies to the sponsor/customer based on software testing and/or software reliability model results. 93 Lesson 1 4 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  94. 94. Technical Engineering Specialties† § Collaborating with technical Specialties – Obtains support and resources from the sponsor/customer for studies and engineering efforts requiring specialized expertise. – Recommends appropriate specialists, communicates pertinent information to the sponsor/customer, and helps to build consensus among key stakeholders. 94 4 Lesson 1 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  95. 95. Collaboration† § Build trust § Build successful team § Communicating the Impact § Being persuasive and influencing outcomes § Facilitating, managing, and championing change § Setting quality standards § Focused on results 95 5 Lesson 1 Roles † MITRE Corporation Systems Engineering Competency Model, http://www.mitre.org/publications/technical-papers/systems-engineering-competency-model
  96. 96. Enough Of Theory Let’s … 96 … and actually start DOING Systems Engineering Lesson 1
  97. 97. Preparing the SEP and SEMP 97 Lesson 1
  98. 98. SEP and SEMP 98 System Engineering Plan System Engineering Management Plan What Who When Why What Purpose When How The SEM and SEMP are designed to work together. The SEM answers SE questions related to what, how, and when, while the SEMP answers SE questions related to what, who, when, and why (i.e., why a particular organization or program is implementing or not implementing a particular SE element versus the SEM discussion regarding an SE element’s purpose). The what, or products and activities of SE, directly connects the SEM and SEMP, and the when provides a secondary correlation. Lesson 1
  99. 99. SEP and SEMP Needed for Success System Engineering Plan § What needs to happen from the customer perspective § Approach to key areas – Requirements – Technical Staffing and Organizational Planning – Technical Baseline Management – Technical Review Planning – Integration with Program Management § Provides contractor guidance for Systems Engineering applied to program System Engineering Management Plan § Response to SEP and contract § Defines contractor technical planning processes § Articulates details of – Processes – Tools – Organization § Describes activities involved in transforming requirements into deliverables § Includes subcontract management 99 Lesson 1
  100. 100. What is the Systems Engineering Plan?† § Articulates and communicates technical planning and management approach to program team, stakeholders, and contractor teams (including bidders if provided with Request for Proposal (RFP)). § Captures integration of both government and contractor Systems Engineering (SE) activities, roles, and responsibilities over the acquisition and sustainment life cycle. § Provides expected management interactions and impacts of their respective processes not only by addressing program-tailored processes, but also the "who, when, and to what result(s)” 100 † How to Prepare a Systems Engineering Plan (SEP) that Works, Sharon Vannucci and Lisa Reuss, Systems and Software Engineering Office of the Deputy Under Secretary of Defense for Acquisition and Technology, ODUSD(A&T) 2009 INCOSE SE Summit March 4, 2009 Lesson 1 SEP
  101. 101. Traits of the SEP† § Defines government (customer) technical planning expectations – What needs to happen from customer perspective § Describes overall approach in key areas – Requirements – Technical Staffing and Organizational Planning – Technical Baseline Management – Technical Review Planning – Integration with Program Management § Provides contractor guidance for Systems Engineering as applied to the acquisition program at hand § Identifies to program management and contract personnel the essential Systems Engineering activities and products required 101 Lesson 1 † Defense Acquisition Guide, 16 September 2012, Section 4, Systems Engineering SEP
  102. 102. Preparing the SEP† Program Technical Requirements Engineering Resources and Management Technical Activities and Products 102 1 2 3 † Systems Engineering Plan (SEP) Outline, 20 April 2011 Version 1.0, 04/20/2011, OPR: ODASD (Systems Engineering) Lesson 1 SEP
  103. 103. Program Technical Requirements § Architectures and Interface Control – Program’s DODAF architecture development efforts. – A system physical architecture diagram (delineating physical interfaces), if available. – A system functional architecture diagram (delineating functional interfaces), if available. – How software architecture priorities will be developed and documented. – How architecture products are related to requirements definition. – How engineering and architecture activities are linked. § Technical Certifications – Current technical and compliance certifications 103 1 Lesson 1 SEP
  104. 104. Engineering Resources and Management § Technical Schedule and Schedule Risk Assessment § Engineering Resources and Cost/Schedule Reporting § Engineering and Integration Risk Management § Technical Organization § Relationships with External Technical Organizations § Technical Performance Measures and Metrics 104 2 Lesson 1 SEP
  105. 105. Technical Activities and Products § Results of Previous Phase SE Activities § Planned SE Activities for the Next Phase § Requirements Development and Change Process § Technical Reviews § Configuration and Change Management Process § Design Considerations § Engineering Tools 105 3 Lesson 1 SEP
  106. 106. Traits of the SEMP† § Responsive to the contract and the SEP § Defines contractor (supplier) technical planning – How it will be accomplished from the contractor perspective § Contractor further develops planning outlined in the SEP § Project (Supplier) team articulates details of their – Processes – Tools – Organization – etc. § Describes activities involved in the transformation from requirements to solution § Includes integration of subcontractor planning 106 † Systems Engineering Plan and Systems Engineering Management Plan Alignment, NDIA 11th Annual Systems Engineering Conference October 21, 2008 Lesson 1 SEMP
  107. 107. What is the Systems Engineering Management Plan (SEMP)? The System Engineering Management Plan (SEMP) establishes the overall plan for the system engineering management of the project. The SEMP identifies and describes the project organization, roles and responsibilities, overall tasks, and engineering management planning required to control the design, development, fabrication, and tests associated with the project. 107 Lesson 1 SEMP
  108. 108. Preparing the Systems Engineering Management Plan (SEMP)† Technical Program Planning and Control System Development Process Engineering specialty Integration 108 Lesson 1 1 2 3 SEMP † INCOSE Systems Engineering Handbook v. 3.2, INCOSE-TP-2003-002-03.2, January 2010, §5.1.2.2
  109. 109. Technical Program Planning and Control § Systems Engineering Organization § Systems Engineering IPT interfaces § Technical Risk Analysis § Engineering methods § Program and Technical Reviews § Interface control 109 Lesson 1 SEMP 1
  110. 110. Systems Development Processes § Integrated Product Review Process § Systems Engineering Processes § System Design Processes § Change Control Processes 110 Lesson 1 SEMP 2
  111. 111. Specialty Engineering Integration† § Test & Evaluation § Software Engineering § Integrated Logistics Support § Design Engineering § Manufacturing and Producibility § Quality Assurance § Reliability and Maintainability § Spectrum Management § Concept Development § Architecture Engineering § System Safety Engineering § Acquisition Systems Protection & International Program Security § Survivability § Human Systems Integration § Mass Properties § EMI/EMC § Parts, Materials & Processes § Information Assurance § Netcentric Engineering § Environmental Engineering § Prognostics, Diagnostics Health Management 111 Lesson 1 SEMP 3 † SMC Specialty Systems Engineering, Specialty Engineering Disciplines, Framework and Descriptions, Space and Missile Systems Center, Volume 2, 1st Edition, 3 October 2011,
  112. 112. SEMP and Technical Plans 112 Program Management Plan (inc IMP/IMS) Systems Engineering Management Plan Software Development Plan Hardware Development Plan Configuration Management Plan Quality Assurance Plan IV&V Plan Lesson 1 SEMP
  113. 113. Other Plans Plans directly flowing from the SEMP § Risk and Opportunity Management Plan § Configuration Management Plan Other plans related to the SEMP § Material Management Plan § Subcontractor Management Plan § Property Management Plan § Mission Assurance Implementation Plan § Measurements and Analysis Plan § Software Development Plan § Facilities Management Plan § System Security Management Plan 113 Lesson 1
  114. 114. All This Guidance Needs to Connect with Business Benefits for it to actually be useful 114 Lesson 1
  115. 115. Benefits of Applying Systems Engineering† § SE ensures the effective development and delivery of capability through the implementation of a balanced approach with respect to cost, schedule, performance, and risk using integrated, disciplined, and consistent activities and processes regardless of the acquisition life cycle. § SE enables the development of engineered resilient systems that are trusted, assured, and easily modified (agile). § SE planning identifies the most effective and efficient path to deliver a capability, from identifying user needs and concepts through delivery and sustainment. § SE event-driven technical reviews and audits assess program maturity and determine the status of the technical risks associated with cost, schedule, and performance goals. 115 † Defense Acquisition Guide 16 September 2013 Lesson 1
  116. 116. Benefits of Applying a SE Framework† § Focuses systems management across the life cycle by providing: – Insight into what should be assessed – Holistic view of engineering a system (software, hardware, humans, and processes) – A process framework that: • Is easy to tailor to meet project and organization needs • Reduce development risk – A basis for • Stage-gate life cycle models including agile development • Communication between all stakeholders • Coordination of work processes • Integration of management agreements with technical management processes 116 † Adapted from ISO/IEC JTCI/SC7/WG7 Lesson 1
  117. 117. Numbers Talk 117 Lesson 1
  118. 118. Critical Success Factors Before Proceeding to Lesson 2 § Principles are Immutable § Practices are many times domain specific § Processes tailored to business need § Systems Engineering Management Plan (SEMP) states how Practices and Processes are implemented on specific programs 118 Lesson 1
  119. 119. Another Critical Success Factor† § Systems Engineering Management Plan (SEMP) is the supplier plan to conduct, manage, and control of the integrated engineering effort. – The SEMP States how the program will reach DONE § Systems Engineering Plan (SEP) is the acquirer’s technical planning document required for milestone approval on the program. – The SEP States what DONE looks like § Keeping these aligned is a Critical Success Factor for all programs applying Systems Engineering. † Systems Engineering Plan and Systems Engineering Management Plan Alignment NDIA 11th Annual Systems Engineering Conference October 21, 2008, Chet Bracuto DoD OUSD A&T (SSE) Bob Scheurer P.E., P.M.P. Boeing Integrated Defense Systems 119 Lesson 1
  120. 120. Lesson 2 The Immutable Principles of System Engineering These principles are applicable to all domains, all technologies, all businesses, services or outcomes and all missions or products 120
  121. 121. Principles of Systems Engineering† Concept Development Requirements Engineering System Architecture System Design and Development System Integration Test and Evaluation Transition to Operations and Maintenance 121 Lesson 2 1 2 3 4 6 5 † MITRE Systems Engineering Guide, http://www.mitre.org/publications/systems-engineering-guide/systems-engineering-guide 7
  122. 122. Concept Development § Operation Needs Assessment § Concept of Operations § Operational Requirements § High-Level Conceptual Definition 122 Lesson 2 1 Concept Development
  123. 123. Operational Needs Assessment § Determine the specific requirements of the needs assessment process that apply. § Identify specific stakeholders in needs assessment process. § Identify and obtain support from operational or capability domain experts. § Develop a needs assessment plan and schedule, including scenario-driven experiments, gap analysis, tradeoff studies. § Identify analytical tools necessary to define and quantify needs. § Implement and conduct needs assessment. 123 1 Concept Development Lesson 2
  124. 124. Concept of Operations § The objective of the ConOps is to – Provide end-to-end traceability between operational needs and captured source requirements. – Establish a high-level basis for requirements that supports the system over its life cycle. – Establish a high-level basis for test planning and system-level test requirements. – Support the generation of operational analysis models (use cases) to test the interfaces. – Provide the basis for computation of system capacity. – Validate and discover implicit requirements. 124 1 Concept Development Lesson 2
  125. 125. Concept of Operations (ConOps) § Contents of the ConOps includes – The existing system (manual or automated) the user wants to replace. – Justification for a new or modified system (including restrictions on that system). – A description of the proposed system. – Scenarios highlighting use of the system in the user's environment including internal and external factors. 125 1 Concept Development Lesson 2
  126. 126. Operational Requirements § The operational requirements must answer: – Who needs this requirement? – Who will be operating the system? – What functions/capabilities must the system perform? – Where will the system be used? – When will the system be required to perform its intended function and for how long? – How will the system accomplish its objective? – How will the requirements be verified? 126 1 Concept Development Lesson 2
  127. 127. Operational Requirements § Developing the Operational Requirements – Identify stakeholders – Elicit requirements for what the system must accomplish and how well. – Define constraints imposed by agreements or interfaces – Establish critical and desired user performance – Establish measures of effectiveness and suitability 127 1 Concept Development Lesson 2
  128. 128. High-Level Conceptual Design § Main objectives of the system § Who will use the resulting system § What are the properties of the resulting system § What capabilities does the system provide § What other systems or processes does this system interface with 128 1 Concept Development Lesson 2
  129. 129. Requirements Engineering § Analyze and define requirements § Elicit, collect, and develop requirements § Address uncertainty in requirements 129 2 Lesson 2
  130. 130. A Requirement is …”A statement identifying a capability, a physical characteristic, or a quality factor that bounds a product or process need for which a solution will be pursued.” – IEEE Standard 1220–2007–05–15 The hardest single part of building a system is deciding what to build … ... No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later. – Fred Brooks “No Silver Bullet,” 1987 What Is a Requirement? 130 Performance-Based Project Management(sm), Copyright © Glen B. Alleman, 2012, 2013 Lesson 2 2 Requirements Engineering
  131. 131. We’ve All Been Here Before 131 I want it over my ears Got it 2 Lesson 2 Requirements Engineering
  132. 132. Steps in the Requirements Development Process § Perform Fact finding – Identify and capture needed capabilities § Gather and Classify Requirements – Capture requirements needed to implement capabilities § Prioritize Requirements – Allocate requirements to appropriate components in the system hierarchy § Integrate and Validate Requirements – Establish methods for verification of each requirement – Ensure product complies with requirements to deliver capabilities 132 Lesson 2 2 Requirements Engineering
  133. 133. Performance Fact Finding § Produce an overall statement of the problem in the operational context. § Develop the overall operational and technical objectives of the target system. § Defined the boundaries and interfaces of the target system. 133 Lesson 2 2 Requirements Engineering
  134. 134. Gather and Classify Requirements § Gather required system capabilities, functional, nonfunctional and environmental requirements, and design constraints. § Build the Top Down capabilities and functional decomposition of the requirements in a Requirements Management System. 134 Lesson 2 2 Requirements Engineering
  135. 135. Evaluate and Rationalize Requirements § Answer the question “why do I need this?” in terms of operational capabilities. § Build a cost / benefit model using probabilistic assessment of all variables, their dependencies, and impacts. § For all requirements, perform a risk assessment to cost and schedule. 135 Lesson 2 2 Requirements Engineering
  136. 136. Prioritize Requirements § Determine criticality for the functions of the system. § Determine trade off relationships for all requirements to be used when option decisions must be made. § For all technical items, prioritize their cost and dependency. 136 Lesson 2 2 Requirements Engineering
  137. 137. Integrate and Validate Requirements § Address the completeness of requirements by removing all “TBD” items. § Validate that the requirements are traceable to system capabilities, goals, and mission. § Resolve any requirements inconsistencies and conflicts 137 Lesson 2 2 Requirements Engineering
  138. 138. Systems Architecture § Architecture Development § Architecture Frameworks – U.S. Department of Defense Architecture Framework (DoDAF) – The Open Group Architecture Framework (TOGAF) – Object-oriented with Unified Modeling Language – Spewak architecture process and Zachman Framework 138 Lesson 2 3
  139. 139. Architecture Development § Define the architecture purpose, value, and decisions it will support. § Create, refine, and update the architecture in an iterative way throughout the acquisition life cycle. § Validate the architecture will meet expectations when implemented. § Define roles for team members to guide and coordinate their efforts. § Create estimates and schedules based on the architectural blueprint. § Use the architecture to gain insight into project performance. § Establish a lightweight, scalable, tailorable, repeatable process framework 139 Systems Architecture 3 Lesson 2
  140. 140. System Design and Development § Assessing Design’s Ability to Delivery the Needed Capabilities defined in the technical and operational requirements. § System-Level Technical Requirements § Top-Level System Design 140 Lesson 2 4
  141. 141. Systems Integration § Integration Testing § Develop and Evaluate Integration and Interoperability § Identify and Assess Integration and Interoperability § Interface Management 141 Lesson 2 5
  142. 142. Test and Evaluation § Test and Evaluation Plans and Procedures § Certification Patterns § Test and Evaluation § Implementation, O&M, and Transition § Verification and Validation 142 Lesson 2 6
  143. 143. Transition to Operations and Maintenance† § Evaluate the end product, personnel, and enabling product readiness for product transition § Prepare the end product for transition § Transition the end product to the customer with required documentation based on the type of transition required § Prepare sites, as required, where the end product will be stored, assembled, integrated, installed, used, and/or maintained § Capture product implementation work products 143 Lesson 2 7 † NASA Systems Engineering Handbook, SP-2007-6105
  144. 144. Lesson 3 System Engineering Practices Practices put the Principles to work. Without Practices, Principles are just nice book learning waiting for a problem to solve 144
  145. 145. Systems Engineering Practices† Product Implementation Product Integration Product Verification Product Validation Cross Cutting Technical Management General Management 145 Lesson 3 1 2 3 4 5 6 †NASA Systems Engineering Handbook, SP-2007-6105
  146. 146. Systems Engineering Practices Live in the Vee 146 We’ll develop the Processes of these in Lesson 4 after we development the Practices Lesson 3
  147. 147. Systems Engineering Management Plan (SEMP) Is Home For Practices The System Engineering Management Plan (SEMP) describes the efforts for planning, controlling and conducting a fully integrated engineering effort. The SEMP is used to understand and evaluate the engineering work efforts as part of the program monitoring process. 147 Lesson 3
  148. 148. Two Primary Practices Product Realization § Implementation § Integration § Verification § Validation § Transition Technical Management § Technical Planning § Requirements Management § Interface Management § Risk Management § Configuration Management § Data Management § Technical Assessment § Decision Analysis 148 Lesson 3
  149. 149. PRODUCT REALIZATION PROCESS The product realization processes are applied to each operational/mission product in the system structure starting from the lowest level product and working up to higher level integrated products. 149 Lesson 3
  150. 150. Product Implementation† § Prepare to conduct implementation § Purchase, Make, Reuse – Purchase from commercial vendor – Make or develop product – Reuse existing product § Capture work products – Drawings – Designs – Model descriptions 150 Lesson 3 1 † NASA Systems Engineering Handbook, SP-2007-6105
  151. 151. Product Integration† § Prepare production integration strategy – Detail planning – Integration sequences and procedures – Product configuration § Prepare lower level products § Prepare integration environments § Assemble and integrate products into end product § Conduct functional testing § Prepare product support documentation § Capture work product 151 Lesson 3 2 † NASA Systems Engineering Handbook, SP-2007-6105
  152. 152. The Right Product That Does The Right Things Verification The assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. Validation The evaluation if the product, service, or system complies with a regulation, requirement, specification, or imposed condition. It is often an internal process. 152 Lesson 3 3 4
  153. 153. Product Verification† § Verification planning § Verification preparation § Conduct verification § Analyze verification results § Capture verification work products 153 Lesson 3 3 † NASA Systems Engineering Handbook, SP-2007-6105
  154. 154. Types of Verification† § Analysis – Modeling or analytical techniques to predict suitability of a system to meet stakeholder expectations § Demonstration – Showing the use of product achieves specified requirements § Inspection – Visual examination of end product § Testing – Use of the end product to obtain detailed date needed to verify performance 154 Lesson 3 3 † NASA Systems Engineering Handbook, SP-2007-6105
  155. 155. Product Validation† § Validation planning § Perform Validation § Analyze outcomes of validation § Report validation outcomes § Capture work products 155 Lesson 3 4 † NASA Systems Engineering Handbook, SP-2007-6105
  156. 156. TECHNICAL MANAGEMENT Are used to manage the technical development of the system increments, including the supporting or enabling systems. Consist of: Technical Planning, Requirements Management, Interface Management, Risk Management, Configuration Management, Technical Data Management, Technical Assessment and Decision Analysis. 156
  157. 157. Cross Cutting Technical Management† § Technical Planning § Requirements Management § Interface Management § Technical Risk Management § Configuration Management § Technical Data Management § Technical Assessment § Decision Analysis 157 Lesson 3 5 † NPR 7123.1B NASA Systems Engineering Processes and Requirements
  158. 158. General Management § Planning § Cost And Schedule § Subcontractor Management § Integration Engineering § Communications § Risk And Opportunity § Decision Processes § Program Reviews 158 Lesson 3 6
  159. 159. General Management Planning § Program planning – Establish PMB – Identify on-ramps/off-ramps – Develop PM plans § Execution management – Maintain baselines § Execution monitoring – Technical performance measures – Earned value management – Risk and opportunity management § Execution control – Analysis and assessment – CCB/ERB 159 Lesson 3 6
  160. 160. General Management Cost and Schedule 160 Program Manager § Establish initial program baseline § Develop cost control process (EVM) Execution Management § Execute processes § Maintain baselines § Use CAIV in Trade Studies and Decision making Monitor § Take performance measures § Conduct cost review Control § Analyze and Assess § Corrective actions § Architecture and PM decisions based on study results § Anticipate future trends Status Lessons Learned Processes Baselines Actions Lesson 3 6
  161. 161. General Management Risk and Opportunity 161 Risk and Opportunity Management Program Manager Chief Engineer Customer Operations Manager Subcontract Manager Risk and Opportunity Manager Lesson 3 6
  162. 162. Roles and Responsibilities in Risk and Opportunity Management § Risk and Opportunity Manager – Process owner – Assess mitigation processes – Risk forums and meetings § Program Manager – Chairs ROMB – Provides resources – Validates scope impacts – Ensures compliance across program, § Chief engineer – Supports analysis – Supports mitigation and realization plans – Reviews TPM plans 162 Lesson 3 6
  163. 163. Program Risk Analysis† § Program risks are elicited from: – Requirements – Technology – Logistics – Management – Schedule 163 † Systems Engineering Management Plan (SEMP) for the Unmanned Control & Tracking Systems (UCATS), George Mason University, Department of Systems Engineering and Operations Research, SYST 798 Applied Project Course, Fall 2009 6 Lesson 3
  164. 164. Lesson 4 System Engineering Processes Processes are how the Principles and Practices are applied to a specific problem. In this course, we’ll apply them to our UAV Program. 164
  165. 165. Systems Engineering Processes Agreement Processes Enterprise Processes Project Management Processes Technical Management Processes 165 Lesson 4 2 3 4 1
  166. 166. AGREEMENT PROCESSES Specifies the requirements for the establishment of agreements between organizations 166 1 Lesson 4
  167. 167. Agreement Processes § Acquisition processes § Supply processes 167 Lesson 4
  168. 168. Acquisition Processes § Establish a plan for how the acquisition will be conducted. § Prepare a request for the supply of a product or service. § Communicate the request for the supply of a product or service to identified suppliers. § Select a supplier. § Negotiate an agreement with the supplier. § Assess the execution of the agreement. § Confirm that the delivered product or service complies with the agreement. § Make payment or provide other agreed consideration to the supplier for the product or service rendered. 168 Lesson 4
  169. 169. Supply Processes § Determine the existence and identity of an acquirer who has, or who represents a party or parties having a need for a product or service. § Evaluate a request for the supply of a product or service to determine feasibility and how to respond. § Prepare a response that satisfies the solicitation. § Negotiate an agreement with the acquirer. § Execute the agreement according to the Supplier’s established project plans and in accordance with the agreement. § Assess the execution of the agreement. § Deliver the product or service in accordance with the agreement criteria. § Accept and acknowledge payment or other agreed consideration. § Transfer the responsibility for the product or service to the acquirer, or other party, as directed by the agreement. 169 Lesson 4
  170. 170. ENTERPRISE PROCESSES Enterprise processes manage the organization’s capability to acquire or supply products or services throughout the lifecycle. Enterprise processes provide resources and infrastructure to support projects and ensure satisfaction of organizational objectives and established agreements. 170 2 Lesson 4
  171. 171. Enterprise Processes § Environment management § Investment management § Life cycle management § Resource management § Quality management 171 Lesson 4
  172. 172. PROJECT MANAGEMENT PROCESSES Managing the project and assessing the technical and operational performance starts with a Systems Engineering process. Using Measures of Effectiveness, Measures of Performance, and Technical Performance Measures – developed in Lesson 4 - the project status presents Physical Percent Complete in units of measure meaningful to the decision maker. This is the process needed to increase the probability of project success. 172 3 Lesson 4
  173. 173. Project Management Processes § Planning § Assessment § Project Controls § Decision-making § Risk Management § Configuration Management § Information Management 173 Lesson 4
  174. 174. Performance Targets Are Required Before Performance Measures Can Be Useful 174 Lesson 4
  175. 175. Project Management Processes § Identifying what “done” looks like § Planning and schedule the work to reach “done” § Identifying impediments to reaching “done” § Measuring progress toward “done” § Taking corrective action to arrive on-time, on- budget, and deliver needed capabilities to the customer 175 Lesson 4 4
  176. 176. Event Based Planning § SRR (Systems Requirements Review) § SFR (System Functional Review) § PDR (Preliminary Design Review) § CDR (Critical Design Review) § ASFUT/GSFUT (Air System/Ground System Functional Unit Test) § TRR (Test Readiness Review) § SVR (System Validation Review) § PRR (Production Readiness Review) 176 Lesson 4
  177. 177. INCOSE VEE and Our IMP/IMS 177 Combine DT&E/O Demonstration` System to Specified User Needs and Environmental Constraints Interpret User Needs, Refine System Performance Specifications, and Environmental Constraints SRR Develop System Functional Specifications and System Verification Plan SFR Evolve Functional Performance Specifications into CI Functional (Design To) Specification and CI Verification Plans PDR System DT&E, Verify System Functionality & Constraints Compliance to Specifications TRR Integrated DT&E, Verify Performance Compliance to Specifications CI Verification DT&E Evolve Functional Performance Specifications into Product (Build To) Documentation and Verification Plans CDR Fabricate, Assemble, Unit Test to Build To Documentation Individual CI Verification DT&E ASFUT GSFUT System Decomposition System Realization System Development and Demonstration SVR PRR Lesson 4 4
  178. 178. Different Levels of Detail in the UAV Model and Associated Activities† 178 † System Testing in the Avionics Domain, von Aliki Ott, zur Erlangung des Grades einer Doktorin der Ingenieurwissenschaften, Vorgelegt im Fachbereich 3 (Mathematik & Informatik) der Universit¨at Bremen im Oktober 2007 Lesson 4 4
  179. 179. Connecting The Dots for the Project Management Processes § Integrated Master Plan and Integrated Master Schedule vertically and horizontally traceable. § The left side of the Vee and the right side are connected by the IMP program events. 179 4 Lesson 4
  180. 180. Quick View to IMP/IMS § Vertical traceability defines the increasing maturity of key deliverables § Horizontal traceability defines the work activities needed to produce this increasing maturity § Both are needed, but the vertical traceability is the starting point § Program Events, Significant Accomplishments, and Accomplishment Criteria must be defined before the horizontal work activities can be identified § For all IMP elements, Key Risks must be identified and assigned from Day One, even without mitigations 180 4 Lesson 4
  181. 181. A Critical Understanding of the IMP The IMP defines the connections between the Product maturity – Vertical – and the implementation of this Product maturity through the Functional activities – the Horizontal 181 4 Lesson 4
  182. 182. The 1st Principle IMP Building is a Full Contact Sport 182 4 Lesson 4
  183. 183. TECHNICAL MANAGEMENT PROCESSES Systems Engineering provides the Programmatic framework for the success of the project. Technical disciplines produce the products and services, within that framework that enable the needed capabilities to be delivered on time, on budget, that meet the requirements of the customer 183 Lesson 4 4
  184. 184. Technical Management Processes Work Breakdown Structure (WBS) Integrated Master Plan (IMP) Integrated Master Schedule (IMS) Cost Basis of Estimate (BoE) Risk Management Plan (RMP) 184 Lesson 4 1 2 3 4 5
  185. 185. WORK BREAKDOWN STRUCTURE The WBS is a product-oriented family tree composed of hardware, software, services, data, and facilities. The family tree results from Systems Engineering efforts during the acquisition of a defense materiel item. A WBS displays and defines the product, or products, to be developed and/or produced. It relates the elements of work to be accomplished to each other and to the end product. In other words, the WBS is an organized method to breakdown a product into sub-products at lower levels of detail.† 185 † MIL-STD-881C, Work Breakdown Structure for Defense Materiel Items, 3 October 2011, AMSC 9231 Lesson 4 1
  186. 186. § Defines the total System products and services that produce those products § Provides the framework for planning, prioritizing, managing, and tracking all work done on the program – Products – Supporting services – Facilities § Provides the framework for – Cost Structure for estimating and cost reporting – Resource allocation – Status Reporting – Performance Measurement – Identify and managing program risk 186 What is the Work Breakdown Structure? Lesson 4 1
  187. 187. Physical Architecture and the WBS 187 Lesson 4 1
  188. 188. Systems Architecture Drives the Work Breakdown Structure 188 Lesson 4 1
  189. 189. § End Products used to implement the mission – Based on the System (Physical) Architecture § Enabling Products and Services – Products and services required to develop, produce, and support the end items – Based on Life Cycle § The first three (end product) WBS levels: – Level 1: Overall System – Level 2: Major Element (or Segment) – Level 3: Subordinate Components (or Prime Items) § Levels 4+ continue the decomposition to the Configuration Item (CI) level 189 Structure of our UAV WBS Lesson 4 1
  190. 190. Three WBS’s for Our UAV Program WBS § High-Level (First 3 levels) § Provides Program Structure § Generally developed/controlled by Customer (Government) Contract WBS § Detailed (Levels 4+) § Provides framework for Contract Work Packages and Costing § Generally developed by Contractor § Generally follows Program WBS 190 Subcontract WBS § Detailed (Level 4+) § Provides framework for Subcontract Work Packages and Costing § Generally developed by Subcontractor § Generally follows Contract WBS Lesson 4 1
  191. 191. § Systems Engineering generally provides structure for: – Levels 1-3 of all WBS based on MIL-HDBK-881C – Levels 4+ of the End Product and Systems Engineering portions of the WBS § Program Control and suborganization participation – Develop Project Management and SE portions of WBS – Develop other parts of WBS as appropriate SE support – Oversee entire WBS development since it serves as framework for project control and management (costing, resourcing, reporting, risk, etc.) § Design and Development IPT participation (development of End Product portion of WBS) § OT&E participation (development of T&E portion of WBS) § Other suborganization participation (to ensure all work activities are properly represented) How is the WBS Built? If we do this By the book, Systems Engineering generally leads WBS development, since the WBS should be based on the system (physical) architecture But in reality, Program Controls often develops the WBS with SE support 191 Lesson 4 1
  192. 192. Using a Generic UAV WBS to Capture Technical Architecture 192 § Technical performance for each subsystem with impacts on cost, schedule, and technical performance are defied in the WBS Dictionary. § Each risk held in the Risk Register, marked with WBS. § WBS number used to trace the risk to the IMS, PE, SA, AC, CA, WP. § Risk retirement plans can then be shown by PE or Milestone in a risk waterfall Lesson 4 1
  193. 193. Level 2 Unmanned Air Vehicle (UAV) 193 Lesson 4 1
  194. 194. Level 3 UAV Avionics 194 Lesson 4 1
  195. 195. Level 4 UAV Avionics – GN&C 195 Lesson 4 1
  196. 196. What’s Our Goal? 196 Count Cows without having to go count cows Lesson 4
  197. 197. § The WBS is an essential tool for the organization and coordination of Systems Engineering processes, and is a product of the Systems Engineering process § The importance of the WBS extends to business professionals and contracting officers. § The needs of all stakeholders must be considered in its development § The Program Office develops the Program WBS (and a high level contract WBS for each contract). Each contractor develops the lower levels of the Contract WBS for their contract § The System (Physical) Architecture provides the basic structure for the “Product” part of the WBS § SOW tasks should flow from the WBS § The WBS provides a structure for organizing IPTs and tracking metrics 197 Summary of the WBS Lesson 4 1
  198. 198. INTEGRATED MASTER PLAN The IMP is an event-based plan consisting of a hierarchy of program events, with each event being supported by specific accomplishments, and each accomplishment associated with specific criteria to be satisfied for its completion. 198 2 Lesson 4
  199. 199. 199 The IMP tells us where is the program going? The Plan describes where we are going, the various paths we can take to reach our destination, and the progress or performance assessment points along the way to assure we are on the right path. These assessment points measures the “maturity” of the product or service against the planned maturity. This is the only real measure of progress – not the passage of time or consumption of money. The Integrated Master Plan (IMP) Is A Strategy For The Successful Completion Of The Project Lesson 4
  200. 200. Integrated Master Plan § Vertical traceability defines the increasing maturity of key deliverables § Horizontal traceability defines the work activities needed to produce this increasing maturity § Both are needed, but the vertical traceability is the starting point § Program Events, Significant Accomplishments, and Accomplishment Criteria must be defined before the horizontal work activities can be identified § For all IMP elements, Key Risks must be identified and assigned from Day One, even without mitigations 200 Lesson 4 2
  201. 201. Risk Management Building the IMP Starts at the RFP 201 SOO ConOps SOW Technical and Operational Requirements CWBS & CWBS Dictionary Integrated Master Plan (IMP) Integrated Master Schedule (IMS) Earned Value Management System Objective Status and Essential Views to support the proactive management processes needed to keep the program GREEN Performance Measurement Baseline Measures of Effectiveness Measures of Performance Measures of Progress Key Performance Parameters Program Specific Key Performance Parameters Technical Performance Measures WBS CWBS Lesson 4 2
  202. 202. The IMP / IMS Structure 202 IMS IMP Describes how program capabilities will be delivered and how these capabilities will be recognized as ready for delivery Supplemental Schedules Work Packages and Tasks Criteria Accomplishment Events or Milestones Lesson 4 2
  203. 203. The IMP/IMS provides Horizontal and Vertical Traceability of progress to plan § Vertical traceability AC è SA è PE § Horizontal traceability WP è WP è AC 203 Program Events Define the maturity of a Capability at a point in time. Significant Accomplishments Represent requirements that enable Capabilities. Accomplishment Criteria Exit Criteria for the Work Packages that fulfill Requirements. Work Package Work Package Work Package Work Package Work Package Work Package Work package Lesson 4 2
  204. 204. The IMP’s role during Execution 204 Program Execution PMB for IBR Proposal Submittal DRFP & RFP Performance Measurement Baseline Tasks (T) BOE % Complete Statement of Work Program Deliverables IMP Accomplishments (A) Criteria (C) EVMS Events (E) Budget Spreads by CA & WP CAIV Capabilities Based Requirements X BCWS = Probabilistic Risk Analysis = Time keeping and ODC = Technical Performance Measure BCWP ACWP Cost & Schedule Risk Model BCWS Decreasing technical and programmatic risk using Risk Management Methods IMS Physical % Complete Continuity and consistency from DRFP through Program Execution WBS Lesson 4 2
  205. 205. INTEGRATED MASTER SCHEDULE The IMS is an integrated, networked schedule containing all the detailed work packages and planning packages necessary to support the Integrated Master Plan† 205 3 Lesson 4 † Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide, V0.9, October 21, 2005
  206. 206. § The WBS is Paramount § The IMP defines the increasing maturity of the program’s deliverables (end item) § The IMS sequences the Work Packages containing the work activities to produce the End Item Deliverables § The IMS is built from the IMP, with WBS coding to assure coverage of all deliverables 206 Quick View of Building the IMS Lesson 4
  207. 207. The Integrated Master Schedule § The horizontal sequence of work activities that produce increasing maturity of the product or services delivered by the program 207 Lesson 4
  208. 208. GAO Schedule Assessment Guide† tells us to … Capture All Activities 208 1 2 3 4 5 6 7 8 9 10 Sequence These Activities Assign Resources To These Activities Establish Duration For These Activities Verify Schedule Is Traceable Horizontally And Vertically Confirm Valid Critical Path – schedule matches program Ensure Reasonable Total Float Conduct Schedule Risk Analysis Update Schedule With Actual Progress Maintain Baseline with Repeatable Process Lesson 4 † GAO Schedule Assessment Guide GAO-12-120G
  209. 209. Scheduling and Planning Excellence Guide (PASEG) tells us to† … Capture all discrete, authorized work effort in the IMS 1 2 3 4 5 6 7 8 Integrate IMS logic Vertically 1st than Horizontally Assure IMS is complete, traceable, documented Assure accurate progress through the status date Show meaningful critical paths – schedule and program Use IMS for timely and effective decision making Align IMS with actual and projected resource demands Baseline and Maintain IMS using repeatable processes † The PASEG or GAO Guide is not IMP centric, so this advice is edited to focus on the IMP first, then to build the credible IMS. Without the IMP, the sequence of work in the IMS fails to show the increasing maturity of the deliverables through their Measures of Performance, Technical Performance Measures, and Risk Retirement assessment. Start with the IMP, and only then development the IMS. 209 Lesson 4
  210. 210. As Systems Engineers Why Do We Care About the IMS? The planned activities for the project that assure the Measures of Effectiveness (MOE)s, Measures of Performance (MOP), Technical Performance Measures (TPM) all in support of delivered capabilities have to show up on time, on schedule, and actually work for the customer to be satisfied. The IMS tells us in what order to perform the work to make this happen. 210 Lesson 4
  211. 211. This is Where the Action Takes Place 211 Lesson 4
  212. 212. BASIS OF ESTIMATE The Basis of Estimate (BoE) is a description of how cost estimate was obtained for each WBS element for which a cost is estimated 212 4 Lesson 4
  213. 213. The Real Problem 213 Lesson 4
  214. 214. The Cost Estimating Problem 214 I need an estimate for all the software for the GN&C boxes for the proposal, and I need it in 2 weeks Here’s a 80% confidence with a 95% Bayesian interval Hey, none of these intervals turned out to contain the true cost of that software Of course not! This was De Finetti style†, fully coherent representation of your beliefs. They’re not confidence intervals. † In probability theory, de Finetti's theorem explains why exchangeable observations are conditionally independent given some latent variable to which an epistemic probability distribution would then be assigned. It is named in honor of Bruno de Finetti. The variables in the cost estimate are not independent, identically, distributed (i.i.d.) Lesson 4
  215. 215. The following excerpts from an Air Force study done 50 years ago titled: "Predictability of the Costs, Time, and Success of Development" RAND 1959† are revealing: § "To a large extent, the differences which arise do so over the question of the "extent" of the uncertainty in development - over questions like, "Are estimates of cost of production likely to be off by 25 percent or 300 percent?" . . . In this paper, we present the results of some recent research into the extent and nature of the uncertainty in new developments. 1) Early estimates of important parameters are usually quite inaccurate … such estimates are strongly "biased" toward over-optimism … 2) The accuracy of estimates is a function of the stage of development, i.e. estimates improve as development of the item progresses … 215 Before we start … Lesson 4
  216. 216. § Do we know what kind of cost estimate we’re looking for? 216 A bigger problem in cost estimating Accurate And Precise? Accurate But Not Precise? Precise But Not Accurate Neither Accurate Nor Precise Estimating accuracy is an indication of the degree to which the final cost outcome may vary from the single point estimate – Larry Dysert, AACE Technical Board Chairman Lesson 4
  217. 217. RISK MANAGEMENT PLAN “It is moronic to predict without first establishing an error rate for the prediction and keeping track of one’s past record of accuracy” — Nassim Nicholas Taleb, Fooled By Randomness 217 5 Lesson 4
  218. 218. INCOSE Guide Says† § The purpose of the Risk Management Process is to identify, analyze, treat and monitor the risks continuously. § The Risk Management Process is a continuous process for systematically addressing risk throughout the life cycle of a system product or service. It can be applied to risks related to the acquisition, development, maintenance or operation of a system. 218 Lesson 4 5 † INCOSE Systems Engineering Handbook, 3.2, §5.4 Risk Management Processes
  219. 219. § The effectiveness of risk management depends on the people who set it up and coordinate the risk management process § On many program risk management consists only of having a policy and oversight § If we treat red flags as false alarms rather than early warnings of danger this incubates the threats to program success § Group think of dominate leaders often inhibits good thinking about risks 219 Core Elements of Program Risk Management† † Towards a Contingency Theory of Enterprise Risk Management, Anette Mikes Robert Kaplan, Working Paper 13-063 October 17, 2013 Lesson 4 5
  220. 220. A Risk Management Plan § Risk Management Processes § Risk Reporting and Metrics § Risk Transfer § Risk Management Training § Risk Management Process and Data Reviews § Risk Management Assurance Process Map 220 Lesson 4 5
  221. 221. 221 Risk Management is How Adults Manage Projects – Tim Lister, IBM Aleatory Epistemic
  222. 222. § Uncertainty creates the opportunity for risk § Reducing uncertainty may reduce risk § Two types of uncertainty† – One that can be reduced – One that cannot § A risk informed PMB starts with the WBS § 8 steps are needed to build a risk informed PMB 222 Quick View of How to Manage in the Presence of Uncertainty and Risk Risk informed program performance management is the goal † Distinguishing Two Dimensions of Uncertainty, Craig Fox and Gülden Ülkumen, in Perspectives of Thinking, Judging, and Decision Making Lesson 4 5
  223. 223. Assemble a credible WBS and the Integrated Master Plan / Integrated Master Schedule (IMP/IMS) – WBS Dictionary says what will be built – IMP/IMS Narrative says how, where, and what processes are used to built it Assess the aleatory uncertainties in the WBS and IMS Adjust activity durations and sequence to create the needed margin to handle the aleatory uncertainty Assign schedule and cost margin to protect end item deliverables 223 How to Build a Risk Adjusted IMS in 8 Steps 0 1 2 3 Lesson 4
  224. 224. Identify Event Based uncertainties from WBS Dictionary and IMP Narratives Assign these uncertainties to the Risk Register Determine risk retirement plans and place them in the IMS Determine cost and schedule impacts of unmitigated risks and place them in Management Reserve Assemble mitigated aleatory and epistemic uncertainties with the unmitigated epistemic risk into the Total Allocated Budget 224 Building a Risk Adjusted IMS in 8 Steps (Concluded) 4 5 6 7 8 Lesson 4
  225. 225. § Lack of precision about the underlying uncertainty § Lack of accuracy about the possible values in the uncertainty probability distributions § Undiscovered Biases used in defining the range of possible outcomes of project processes § Natural variability from uncontrolled processes § Undefined probability distributions for project processes and technology § Unknowability of the range of the probability distributions § Absence of information about the probability distributions 225 Sources of Uncertainty Lesson 4 5
  226. 226. 226 Uncertainties are things we can not be certain about. Uncertainty is created by our incomplete knowledge; not by our ignorance Lesson 4
  227. 227. § When we say uncertainty, we speak about a future state of an external system that is not fixed or determined § Uncertainty is related to three aspects of our program management domain: – The external world – the activities of the program – Our knowledge of this world – the planned and actual behaviors of the program – Our perception of this world – the data and information we receive about these behaviors 227 Some Words about Uncertainty Lesson 4 5
  228. 228. § Risk has two dimensions – The degree of possibility that an event will take place or occur sometime in the future – The consequences of that event, once it has occurred § The degree of possibility is qualified as the Probability of Occurrence (event based) or a Probability Distribution Function (a distribution of variability's of a random number) § The consequences are usually taken to be undesirable and qualified as the magnitude of harm and the remaining probability of a recurrence of the same risk. 228 Some Words About the Risk That Results from Uncertainty Lesson 4 5
  229. 229. § Naturally occurring uncertainty and its resulting risk, impacts the probability of a successful outcome. What is the probability of making a desired completion date or cost target? 229 All Program Activities have Naturally Occurring Uncertainty § The irreducible statistical behavior of these activities, their arrangement in a network of activities, and correlation between their behaviors creates risk. § Adding margin protects the outcome from the impact of this naturally occurring uncertainty Lesson 4 5
  230. 230. § Uncertainty is present when probabilities cannot be quantified in a rigorous or valid manner, but can described as intervals within a probability distribution function (PDF) § Risk is present when the uncertainty of the outcome can be quantified in terms of probabilities or a range of possible values § This distinction is important for modeling the future performance of cost, schedule, and techncial outcomes of a program 230 Relationship Between Uncertainty and Risk† Lesson 4 5 † Paul Garvey, Probability Methods for Cost Uncertainty Analysis – A systems Engineering Perspective, Marcel Deker, CRC Press, 2000
  231. 231. Areas for Risk Management In Systems Engineering† 231 Area for Risk Engagement Rationale Steps System interface change management Each time a change to an interface is proposed, the change should be evaluated to see if new uncertainties and risks are being introduced. • Work with systems engineering and the change management function to ensure that assessments are made of the risks associated with each proposed interface change at a level appropriate to the scope of the change. • Ensure that doing the assessments and making use of the results in interface change decisions are written into the change management process and any other relevant documents. • Work with the systems engineering and change management functions to ensure that these steps are written into their processes and any other relevant documents. • Provide training and support on an ongoing basis as required and agreed upon. † Why an INCOSE Systems Approach is Needed, Dick Kitterman, Northrop Grumman Sixteenth Annual International Symposium of the International Council On Systems Engineering (INCOSE) 8 - 14 July 2006 Lesson 4 5
  232. 232. Areas for Risk Management In Systems Engineering† 232 Area for Risk Engagement Rationale Steps System test definition and planning It is necessary to understand where there are risks of being able to test a particular requirement, or related group of requirements. • Work with systems engineering to ensure that risks are identified, analyzed and handled as an inherent part of each stage of test definition and planning. • Work with systems engineering to ensure that these steps are written into their processes and any other relevant documents. • Provide training and support on an ongoing basis as required and agreed upon. † Why an INCOSE Systems Approach is Needed, Dick Kitterman, Northrop Grumman Sixteenth Annual International Symposium of the International Council On Systems Engineering (INCOSE) 8 - 14 July 2006 Lesson 4 5

×