Our systems are bigger, more expensive, more visible and have become critical to our economic well being and national defense.
4. How if coupled with a ‘value driven design’ approach a team can reduce the dynamics of reactionary & ad-hoc decision making especially when design rework is required.
We have relied on process – how do we pay attention to empirical evidence? Have we painted ourselves into a process and IT corner?
Research from U Tokyo mid 1990s. Industrial experience. 12 years. 100s of projects. 1000s of models.New actors and architecture leads to surprising demands for coordination – unlike what was previously embedded from years of stability
[Fayol 1916] Fayol, Henri, (in French), Administration industrielle et générale; prévoyance, organisation, commandement, coordination, controle, H. Dunod et E. Pinat, Paris, 1916. English translation, General and industrial management, Pitman, London, 1949[Tuck 1912] Addresses And Discussions At The Conference On Scientific Management held October 12 . 13 . 14 Nineteen Hundred And Eleven, Dartmouth College. The Plimpton Press, 1912[Weber 1924] Weber, Max, The Theory of Social and Economic Organization (1947 translation by. A.H.Henderson and Talcott Parsons), Simon & Schuster, New York, 1924[Simon 1962] Simon, Herbert A., The Architecture of Complexity, Proceedings of the American Philosophical Society, Vol. 106, No. 6. (Dec. 12, 1962), pp. 467-482, 1962[Burton 1995] Burton, R. and Obel B., Strategic Organizational Diagnosis and Design: Developing Theory for Application, Kluwer Academic Publishers, Boston, 1995
Capacity and behavior! The characteristics of human attention and learning. The aggregation of results, rather than just decomposition.
[Malone 1994] Malone T. and Crowston, K., “The Interdisciplinary Study of Coordination”, ACM Computing Surveys, March, Vol. 26 No. 1, pp. 87-119, 1994
Team size, experience, capacity, time zone, differences in coordination behaviors,…
Screenshots from GPD’s TeamPort and a recent industrial case
(the need for information is pent up; if the teams involved have not interacted over years then their shared tacit knowledge is small, and thus demand to interact suddenly very high)
deNeufville and Scholtes, Flexibility in Engineering Design, 2011, MIT Press[Moser 1998] Moser, B., Kimura, F. and Suzuki H., "Simulation of Distributed Product Development with Diverse Coordination Behavior", Proceedings of the 31st CIRP International Seminar on Manufacturing Systems, Berkeley, California, May 1998[Moser 2009] Moser B., Grossmann W., and Murray P., “Simulation & Visualization of Performance across Subsystems in Complex Aerospace Projects”, Proceedings of the 2009 PMI Global Congress, Orlando, Florida, USA, 2009
Complexity Challenges in the Integration of Systems and Organizations Does Systems Engineering need an Overhaul? NASA PM Challenge 2012 1
AgendaComplexity Challenges in the Integration ofSystems and Organizations• Does Systems Engineering Need an Overhaul?• Looking at Complexity from the Outside In• Complexity & Teams• Dialogue 2
Does Systems Engineering Need an Overhaul? Michael C. Lightfoot NASA Langley Research Center, Hampton, VA PM Challenge 2012, Orlando, FL February 22, 2012 3
Systems Engineering is Being Placed Under the MicroscopeThere is a growing number of engineering communities who are askingtough questions about the current practice of Systems Engineering.Tough Questions: Why do the current SE processes, if rigorously applied, not guaranteeus safe, effective, robust systems delivered on time and within budget? What is it about our methods, processes and tools that seems to fail innewsworthy fashion when we attempt to design and build large-scalesystems. Has our SE system somehow evolved to become a system that defiesour control?Why the tough questions now? 4
Systems Engineering TrendsSystem Size and Complexity has increased: One Example*: F-16, 15 Subsystems, 103 Interfaces F-35, 130 Subsystems, 105 InterfacesOrganizations: • Size increase (100’s to 1000’s), • most likely global teams, • different cultures w/ different incentives • multiple companies, • many reporting structures, • sometimes competing incentivesSubsystems that were once modular in design are nowirreducibly entwined (tightly coupled)Many systems are one of a kind (NASA) or limited quantityproductions* Data courtesy of United Technologies Research Center:https://www.fbo.gov/download/9cb/9cb78f01aa9db1fe92e093e786bc6733/Abstraction_Based_Complexity_Management_Final_Report_Dist_A.pdf 5
Increases in Aerospace Systems ComplexityPaul Eremenko, DARPA, META Program 6
Characteristics of Large-Scale Complex Engineered Systems Increased Engineering Complexity Highly-coupled interfaces, many of which are only discovered during integration & testing or system operation. Design Cycles are Longer and More Complicated Significant Cost and Risk Extremely high political and monetary risk Low tolerance for failures or degraded performance Public fear of catastrophic failure is high Limited opportunities to experiment (trial and error) Very Large, Dispersed Engineering Organizations Yet organizations are expected to function synergistically Coordination and data exchanges are greater in frequency and volume of data. Unlike the early days of SE, no one Chief SE is able to keep the entire system view in his/her head. 8
Classes of Engineered Systems (Relative Comparisons, Not Rigorous Definitions)Simple System: Consist of few parts, Small number of interfaces Interactions well understood & well controlled, Typically used as building blocks for more sophisticated parts & componentsComplicated Consist of many parts, components, subsystemsSystem: Moderate to large number of interfaces Interactions/reactions understood for controlled cases Vigilant control required to properly construct V & V is the basis to accept/reject bad parts, components, subsystems Global system behavior is mostly predictable; Part decomposition & analysis leads to reasonable global property predictionsComplex Can possess extreme numbers of parts, components, subsystemsSystem: Extreme numbers of interfaces- sometimes impossible to identify Interactions understood for limited number of highly controlled cases but mostly unknown due to dynamic adaptations Vigilant control often exercised but system sensitivity is nonlinear & dependent on initial conditions (path dependent). Current analysis tools are poor predictors of system behavior Complete system V & V not possible. Global system behavior can be emergent (reductionist approaches fail) 9
Complicated System Example Star Caliber Patek Phillipe mechanical watch. We understand: how it is constructed, the required tolerances, the order of assembly. Each component works in unisonto accomplish a global function: keeptime precisely. We can take a reductionist path todefine the smallest required partsand can further write equations ofmotion to predict the performanceand functionality of the watch. 10
Complex Systems Through a Complexity Science Lens • Dynamical systemsDynamical/non-linear Highly-coupled • System response is non-linear & sensitive to initial conditions • Consist of many parts, components, or subsystems (agents) that interact Adaptive with each other & the environment Can be Self- organizing • They learn & adapt their behaviors to survive If the adaptation strategy is good they continue to exist If the strategy is bad or non-existent they cease to exist Global behaviors happen without a • They can move from an ordered to disordered statecentralized controller unpredictably, and can be self-organizing Reductionist • No centralized controller approaches do not describe global behaviors • Knowledge of the inner workings of each agent typically shed no information about the global behavior/response of the system 11
Examples of Complex SystemsDynamical/non-linear • Ant colonies • Rain forests Highly-coupled • Communities where you live • U.S. Power Grid • The World Wide Web Adaptive • The Stock Marker • Propagation of infectious diseases Can be Self- organizing • The Global Economy (financial system collapse 2008) • The Occupy ?? Protest Groups • Multinational corporations Global behaviors happen without a • The NASA employees and contractors who supported thecentralized controller Constellation Program The various engineering organizations that developed specific flight Reductionist hardware for Pad Abort Activities approaches do not The NASA PM and SE groups that supported Constellation describe global behaviors Complex Systems can be Technical(Engineered), Biological, Social or some combination 12
Domains of Complexity Social Technical Complex Engineering Organizations Socio- Technical Creating Complex Engineered Systems 13
Why is a Complexity Science Framework Important to the SE Community? Current SE processes consists of experientially-based guidance. Although this guidance is tailorable, it is not deterministic. There currently is no theory, nor “science of system engineering” that enables us to predict the efficacy, resilience or robustness of the systems we produce. Our gut tells us that organizations impact the products we create but we have no analytical tools to express the relationship between the two. A complexity science framework encourages us to question the existence of dynamical relationships where we formerly assumed no or linear relationships existed. This includes interactions between social systems and technological systems. Many of the basic tenets/tools of complexity science are quite familiar to engineers that work in dynamical systems (chaos, non-linear behavior, neural networks, genetic algorithms, graphical modeling & simulation tools tools, etc.) 14
What are the building blocks needed to grow a competency in Complex Engineered Systems? ?????? Listen, Share and Solve Explore, Understand, problems across Integrate social systems disciplines & use new complexity into our decision tools in novel ways. making & SE processesHolistic Systems Uncertainty-Based Statistical Thinking &Thinking Modeling and Simulation Probabilistic Uncertainty[ embrace non-linearity ] Tools and Techniques Analysis 15
Potential Domain Infusions Science of Trans-disciplinary Socio-Technical Engineering Systems Science Social Technical Complex Engineering Organizations Socio- Technical Creating Complex Engineered SystemsEngineering of Systems EngineeringActivities 16
NSF/NASA Workshop on Design of Large-scale Complex Engineered Systems February 7-8, 2012 Arlington, Virginia Organizers:Steven McKnight, NSF Vicki Crisp, NASAChristina L. Bloebaum, NSF Anna-Maria McGowan, NASAGeorge Hazelrigg, NSF Michael Lightfoot, NASA Paul Collopy, University of Alabama, Huntsville 17
Workshop OverviewObjective:• Examine the challenges unique to large-scale complex engineered systems• Examine how we can better prepare for a future of growing system complexity?Four Topic Areas Explored:1. New approaches to system complexity by framing it through a ‘complexity science’ lens.2. Current developments in design science and how might they help us in designing within the SE process.3. Awareness of what is known in organization science and how the engineered product is a function of the organization.4. How decision science can provide a more rigorous approach to decision making in large-scale project teams. 18
Who Attended the NSF/NASA Workshop on The Design of Large-Scale Complex Engineered Systems?• A total of ~115 people in attendance• Government: NSF, NASA, DoD (ODASD, AFRL, AFOSR, ONR, NRL, ARL), V-DOT• Academia (25): University of Illinois at Urbana-Champaign, University of Minnesota, George Mason University, University of Maryland, Northwestern University, University at Buffalo – SUNY, Purdue University, Schulich School of Business, York University, North Carolina State University, Georgia Institute of Technology, Pennsylvania State University, Texas A&M University, Oregon State University, Stevens Institute of Technology, Johns Hopkins University, University of Virginia, University of Michigan, University of Florida, Brigham Young University, Massachusetts Institute of Technology, Iowa State University, Stanford University, George Washington University, Mills College• Industry & Others: Lockheed Martin, Boeing, MITRE, SpaceWorks, Global Project Design, Google, NAE and others• Disciplines Represented: Engineering, Social Science, Cognitive Science, Organization Science, Anthropology and Economics 19
My Workshop Takeaways• Systems Engineering as practiced is laden with human decision making which could be enhanced by the understanding & practice of decision science• SE needs to embrace nonlinearity and embrace a future where the systems we build will not be fully testable (within the current practice of V&V).• In order to better design & build large-scale complex engineered systems of the future we need to 1st build better relationships between: – Complexity Science Researchers – Engineering Design Science Researchers – Organizational Science Researchers – Systems Engineers (PM+SE) – Optimization Researchers – S & T Leaders within Government Agencies• Government participant agreed to form a Community of Practice to exploit unique strengths that NSF, NASA, and DoD can bring to the challenge of large-scale complex engineered systems. 20
AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?• Looking at Complexity from the Outside In• Complexity & Teams• Dialogue 21
Looking at Complexity from the Outside Ina fresh look including outside our current processes Ed Rogan NASA PM Challenge February 22, 2012 www.gpdesign.com | email@example.com