Moser lightfoot pmc2012pres


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Eremenko
  • Our systems are bigger, more expensive, more visible and have become critical to our economic well being and national defense.
  • 4. How if coupled with a ‘value driven design’ approach a team can reduce the dynamics of reactionary & ad-hoc decision making especially when design rework is required.
  • We have relied on process – how do we pay attention to empirical evidence? Have we painted ourselves into a process and IT corner?
  • Research from U Tokyo mid 1990s. Industrial experience. 12 years. 100s of projects. 1000s of models.New actors and architecture leads to surprising demands for coordination – unlike what was previously embedded from years of stability
  • [Fayol 1916] Fayol, Henri, (in French), Administration industrielle et générale; prévoyance, organisation, commandement, coordination, controle, H. Dunod et E. Pinat, Paris, 1916. English translation, General and industrial management, Pitman, London, 1949[Tuck 1912] Addresses And Discussions At The Conference On Scientific Management held October 12 . 13 . 14 Nineteen Hundred And Eleven, Dartmouth College. The Plimpton Press, 1912[Weber 1924] Weber, Max, The Theory of Social and Economic Organization (1947 translation by. A.H.Henderson and Talcott Parsons), Simon & Schuster, New York, 1924[Simon 1962] Simon, Herbert A., The Architecture of Complexity, Proceedings of the American Philosophical Society, Vol. 106, No. 6. (Dec. 12, 1962), pp. 467-482, 1962[Burton 1995] Burton, R. and Obel B., Strategic Organizational Diagnosis and Design: Developing Theory for Application, Kluwer Academic Publishers, Boston, 1995
  • Capacity and behavior! The characteristics of human attention and learning. The aggregation of results, rather than just decomposition.
  • [Malone 1994] Malone T. and Crowston, K., “The Interdisciplinary Study of Coordination”, ACM Computing Surveys, March, Vol. 26 No. 1, pp. 87-119, 1994
  • Team size, experience, capacity, time zone, differences in coordination behaviors,…
  • Screenshots from GPD’s TeamPort and a recent industrial case
  • (the need for information is pent up; if the teams involved have not interacted over years then their shared tacit knowledge is small, and thus demand to interact suddenly very high)
  • deNeufville and Scholtes, Flexibility in Engineering Design, 2011, MIT Press[Moser 1998] Moser, B., Kimura, F. and Suzuki H., "Simulation of Distributed Product Development with Diverse Coordination Behavior", Proceedings of the 31st CIRP International Seminar on Manufacturing Systems, Berkeley, California, May 1998[Moser 2009] Moser B., Grossmann W., and Murray P., “Simulation & Visualization of Performance across Subsystems in Complex Aerospace Projects”, Proceedings of the 2009 PMI Global Congress, Orlando, Florida, USA, 2009
  • Moser lightfoot pmc2012pres

    1. 1. Complexity Challenges in the Integration of Systems and Organizations Does Systems Engineering need an Overhaul? NASA PM Challenge 2012 1
    2. 2. AgendaComplexity Challenges in the Integration ofSystems and Organizations• Does Systems Engineering Need an Overhaul?• Looking at Complexity from the Outside In• Complexity & Teams• Dialogue 2
    3. 3. Does Systems Engineering Need an Overhaul? Michael C. Lightfoot NASA Langley Research Center, Hampton, VA PM Challenge 2012, Orlando, FL February 22, 2012 3
    4. 4. Systems Engineering is Being Placed Under the MicroscopeThere is a growing number of engineering communities who are askingtough questions about the current practice of Systems Engineering.Tough Questions: Why do the current SE processes, if rigorously applied, not guaranteeus safe, effective, robust systems delivered on time and within budget? What is it about our methods, processes and tools that seems to fail innewsworthy fashion when we attempt to design and build large-scalesystems. Has our SE system somehow evolved to become a system that defiesour control?Why the tough questions now? 4
    5. 5. Systems Engineering TrendsSystem Size and Complexity has increased: One Example*: F-16, 15 Subsystems, 103 Interfaces F-35, 130 Subsystems, 105 InterfacesOrganizations: • Size increase (100’s to 1000’s), • most likely global teams, • different cultures w/ different incentives • multiple companies, • many reporting structures, • sometimes competing incentivesSubsystems that were once modular in design are nowirreducibly entwined (tightly coupled)Many systems are one of a kind (NASA) or limited quantityproductions* Data courtesy of United Technologies Research Center: 5
    6. 6. Increases in Aerospace Systems ComplexityPaul Eremenko, DARPA, META Program 6
    7. 7. Large- Scale Complex Engineered Systems 7
    8. 8. Characteristics of Large-Scale Complex Engineered Systems Increased Engineering Complexity  Highly-coupled interfaces, many of which are only discovered during integration & testing or system operation. Design Cycles are Longer and More Complicated Significant Cost and Risk  Extremely high political and monetary risk  Low tolerance for failures or degraded performance  Public fear of catastrophic failure is high  Limited opportunities to experiment (trial and error) Very Large, Dispersed Engineering Organizations  Yet organizations are expected to function synergistically  Coordination and data exchanges are greater in frequency and volume of data.  Unlike the early days of SE, no one Chief SE is able to keep the entire system view in his/her head. 8
    9. 9. Classes of Engineered Systems (Relative Comparisons, Not Rigorous Definitions)Simple System:  Consist of few parts,  Small number of interfaces  Interactions well understood & well controlled,  Typically used as building blocks for more sophisticated parts & componentsComplicated  Consist of many parts, components, subsystemsSystem:  Moderate to large number of interfaces  Interactions/reactions understood for controlled cases  Vigilant control required to properly construct  V & V is the basis to accept/reject bad parts, components, subsystems  Global system behavior is mostly predictable; Part decomposition & analysis leads to reasonable global property predictionsComplex  Can possess extreme numbers of parts, components, subsystemsSystem:  Extreme numbers of interfaces- sometimes impossible to identify  Interactions understood for limited number of highly controlled cases but mostly unknown due to dynamic adaptations  Vigilant control often exercised but system sensitivity is nonlinear & dependent on initial conditions (path dependent).  Current analysis tools are poor predictors of system behavior  Complete system V & V not possible.  Global system behavior can be emergent (reductionist approaches fail) 9
    10. 10. Complicated System Example Star Caliber Patek Phillipe mechanical watch. We understand:  how it is constructed,  the required tolerances,  the order of assembly. Each component works in unisonto accomplish a global function: keeptime precisely. We can take a reductionist path todefine the smallest required partsand can further write equations ofmotion to predict the performanceand functionality of the watch. 10
    11. 11. Complex Systems Through a Complexity Science Lens • Dynamical systemsDynamical/non-linear Highly-coupled • System response is non-linear & sensitive to initial conditions • Consist of many parts, components, or subsystems (agents) that interact Adaptive with each other & the environment Can be Self- organizing • They learn & adapt their behaviors to survive If the adaptation strategy is good they continue to exist If the strategy is bad or non-existent they cease to exist Global behaviors happen without a • They can move from an ordered to disordered statecentralized controller unpredictably, and can be self-organizing Reductionist • No centralized controller approaches do not describe global behaviors • Knowledge of the inner workings of each agent typically shed no information about the global behavior/response of the system 11
    12. 12. Examples of Complex SystemsDynamical/non-linear • Ant colonies • Rain forests Highly-coupled • Communities where you live • U.S. Power Grid • The World Wide Web Adaptive • The Stock Marker • Propagation of infectious diseases Can be Self- organizing • The Global Economy (financial system collapse 2008) • The Occupy ?? Protest Groups • Multinational corporations Global behaviors happen without a • The NASA employees and contractors who supported thecentralized controller Constellation Program The various engineering organizations that developed specific flight Reductionist hardware for Pad Abort Activities approaches do not The NASA PM and SE groups that supported Constellation describe global behaviors Complex Systems can be Technical(Engineered), Biological, Social or some combination 12
    13. 13. Domains of Complexity Social Technical Complex Engineering Organizations Socio- Technical Creating Complex Engineered Systems 13
    14. 14. Why is a Complexity Science Framework Important to the SE Community?  Current SE processes consists of experientially-based guidance.  Although this guidance is tailorable, it is not deterministic.  There currently is no theory, nor “science of system engineering” that enables us to predict the efficacy, resilience or robustness of the systems we produce.  Our gut tells us that organizations impact the products we create but we have no analytical tools to express the relationship between the two.  A complexity science framework encourages us to question the existence of dynamical relationships where we formerly assumed no or linear relationships existed. This includes interactions between social systems and technological systems.  Many of the basic tenets/tools of complexity science are quite familiar to engineers that work in dynamical systems (chaos, non-linear behavior, neural networks, genetic algorithms, graphical modeling & simulation tools tools, etc.) 14
    15. 15. What are the building blocks needed to grow a competency in Complex Engineered Systems? ?????? Listen, Share and Solve Explore, Understand, problems across Integrate social systems disciplines & use new complexity into our decision tools in novel ways. making & SE processesHolistic Systems Uncertainty-Based Statistical Thinking &Thinking Modeling and Simulation Probabilistic Uncertainty[ embrace non-linearity ] Tools and Techniques Analysis 15
    16. 16. Potential Domain Infusions Science of Trans-disciplinary Socio-Technical Engineering Systems Science Social Technical Complex Engineering Organizations Socio- Technical Creating Complex Engineered SystemsEngineering of Systems EngineeringActivities 16
    17. 17. NSF/NASA Workshop on Design of Large-scale Complex Engineered Systems February 7-8, 2012 Arlington, Virginia Organizers:Steven McKnight, NSF Vicki Crisp, NASAChristina L. Bloebaum, NSF Anna-Maria McGowan, NASAGeorge Hazelrigg, NSF Michael Lightfoot, NASA Paul Collopy, University of Alabama, Huntsville 17
    18. 18. Workshop OverviewObjective:• Examine the challenges unique to large-scale complex engineered systems• Examine how we can better prepare for a future of growing system complexity?Four Topic Areas Explored:1. New approaches to system complexity by framing it through a ‘complexity science’ lens.2. Current developments in design science and how might they help us in designing within the SE process.3. Awareness of what is known in organization science and how the engineered product is a function of the organization.4. How decision science can provide a more rigorous approach to decision making in large-scale project teams. 18
    19. 19. Who Attended the NSF/NASA Workshop on The Design of Large-Scale Complex Engineered Systems?• A total of ~115 people in attendance• Government: NSF, NASA, DoD (ODASD, AFRL, AFOSR, ONR, NRL, ARL), V-DOT• Academia (25): University of Illinois at Urbana-Champaign, University of Minnesota, George Mason University, University of Maryland, Northwestern University, University at Buffalo – SUNY, Purdue University, Schulich School of Business, York University, North Carolina State University, Georgia Institute of Technology, Pennsylvania State University, Texas A&M University, Oregon State University, Stevens Institute of Technology, Johns Hopkins University, University of Virginia, University of Michigan, University of Florida, Brigham Young University, Massachusetts Institute of Technology, Iowa State University, Stanford University, George Washington University, Mills College• Industry & Others: Lockheed Martin, Boeing, MITRE, SpaceWorks, Global Project Design, Google, NAE and others• Disciplines Represented: Engineering, Social Science, Cognitive Science, Organization Science, Anthropology and Economics 19
    20. 20. My Workshop Takeaways• Systems Engineering as practiced is laden with human decision making which could be enhanced by the understanding & practice of decision science• SE needs to embrace nonlinearity and embrace a future where the systems we build will not be fully testable (within the current practice of V&V).• In order to better design & build large-scale complex engineered systems of the future we need to 1st build better relationships between: – Complexity Science Researchers – Engineering Design Science Researchers – Organizational Science Researchers – Systems Engineers (PM+SE) – Optimization Researchers – S & T Leaders within Government Agencies• Government participant agreed to form a Community of Practice to exploit unique strengths that NSF, NASA, and DoD can bring to the challenge of large-scale complex engineered systems. 20
    21. 21. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?• Looking at Complexity from the Outside In• Complexity & Teams• Dialogue 21
    22. 22. Looking at Complexity from the Outside Ina fresh look including outside our current processes Ed Rogan NASA PM Challenge February 22, 2012 |
    23. 23. What isComplexity  Complexity means different things in different technical and professional contexts  We encounter most of them in practice  A common language accessible to non-experts would be useful Slide 23Global Project Design © 2012
    24. 24.  Complexity as length of a bit stringCompressible Bit (Kolmogorov, 1965) Strings  What is the shortest computer program that will output a given bit string?  Simple: 0101010101010101…  Slightly more complex: 3.14159265358979323846….  A definition of randomness: a string that cannot be compressed to any shorter Slide 24 computer program Global Project Design © 2012
    25. 25.  Complexity as time required to find a solution Complexity of to a computational problem (e.g. factoring aComputation large composite number, scheduling, routing) (Cook, 1971)  If we can verify an answer quickly (time bounded by a polynomial function of the input length), can we also find an answer quickly?  Probably not in all cases.  “P = NP?”. $1 million prize remains unclaimed for solving. Slide 25 Global Project Design © 2012
    26. 26.  Paradox: how can unpredictable behaviorComplexDynamics result from the laws of classical physics?  Examples: celestial mechanics (Poincare, 1895), fluid dynamics (Lorenz, 1963)  Nonlinear terms amplify small differences in boundary or initial conditions  Solutions to compressible Navier-Stokes equations exhibit qualitative changes in behavior with changes in a parameter (e.g. Reynolds number, Mach number) – bifurcation, strange attractors, and chaos. Slide 26Global Project Design © 2012
    27. 27.  Large, highly interconnected networked Othersystems with systems complex  Hybrid (discrete-continuous or digital-analog) dynamics systems  Example: brains.  80 - 100 billion digital-analog/analog-digital converters  Up to 10,000 inputs to a single converter  Emergent behaviors: decision-making, attention Slide 27 Global Project Design © 2012
    28. 28. Example: stock markets ComplexInteractions  Assume decision makers are rational. All available in information about the value of a stock is reflected in its price. Decision- How can stock markets crash? Making  Decision-makers are not always rational (noise traders, prospect theory, risk of arbitrage).  Sometimes, buyers and sellers in the stock market choose not to reveal all of the information that they know about the value of stocks.  Or, decision-makers as a group may have more knowledge than they have as individuals (muddy children puzzle). An event may make this information common knowledge.  When previously hidden information becomes common knowledge, behavior of many (rational) buyers and sellers Slide 28 can (and does) change quickly. Global Project Design © 2012
    29. 29.  Software (Kolmogorov and P = NP?) Summary: What  Digital-analog interconversion (hardware-softwareComplexities interfaces)do we face inEngineering  Nonlinearity (fluids and structures) Systems?  Large systems with many interfaces, dependencies, or couplings  Human decision makers  Sharing or exchange of knowledge and information  Risk and uncertainty Complex systems have elements we haven’t considered in the past Slide 29 Global Project Design © 2012
    30. 30. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?Looking at Complexity from the Outside In• Complexity & Teams• Dialogue 30
    31. 31. Complexity & TeamsWhat multi-disciplinary research showsabout behavior in socio-technical systems Bryan Moser NASA PM Challenge February 22, 2012 | Slide 31
    32. 32. Who is GPD? • Technology Leaders from Complex Global Industries • U. Tokyo: Graduate School of Frontier Sciences The Design of Global Projects • Rapidly prototype and adjust plans • Predict coordination activity • Drive attention to interactions of value GPD’s Methods & Experience • Visual Modeling of integrated socio-technical architecture • Behavior based simulation including global factors • 15 years of case experience in industry: 3/ month globally GPD’s Partnership Agenda • Measures of Coordination by “Humans in the Loop” • Leverage observation and massive sensing • Practicability of new techniques Slide 32Global Project Design © 2012
    33. 33. Models of “organization” have shifted from Shifting centrally controlled mechanical systems to Models of dynamic organisms with distributed,Organization adaptive, and behavior based subsystems.  planning and forecasting, organizing, commanding, coordinating, and controlling (Fayol, 1916)  structure, hierarchy, authority, roles (Weber, 1924)  as systems with boundaries, goals, incentives, behaviors (Simon, 1962)  differentiation, formalization, complexity, centralization, span of control, rules, procedures… (Burton, 1995 and others) Slide 33 Global Project Design © 2012
    34. 34. Work as a Product Development viewed as a Socio- “socio-technical” system if we includeTechnical “Humans in the Loop” System  People do work, process information, and interact as part of an organization  Individuals allocate attention based on behaviors within limited capacity  Organizations with architecture exhibit emergent behavior (e.g. exception handling, quality…) Slide 34Global Project Design © 2012
    35. 35. Engineering  What if an IT system allowed in Socio- ̵ teams with access to all information? technical systems ̵ all processes clearly written? ̵ workflow software to support tasks?  If requirements, work packages, and dependencies are clearly written and assigned, is performance guaranteed? What about human performance during complex work isn’t addressed above? Slide 35 Global Project Design © 2012
    36. 36. Coordination  Coordination is the activity to manage dependencies.  What portion of your weekly effort is spent coordinating?  What happens to items in your inbox when it overflows? Manufacturing has shown for decades that managing human attention is a key: if we over-automate, quality drops Slide 36 Global Project Design © 2012
    37. 37. Three activities to realize aArchitecture subsystem. Three teams. &Coordination  Independent activities. Where will coordination occur?  Dependent activities. Where coordination?  Changed pattern of roles and dependence, yet scope and resources unchanged. Where Coordination? The demand and supply of coordination activity are driven by the integrated architecture of the project. Slide 37 Global Project Design © 2012
    38. 38. Architecture  Teams have structure. & Quality What coordination is this? What impact on performance?  “Exception Handling”  Dependent activities. Why does capacity of Team_1 now matter? And in this case?  What if: Teams in different time zones? Teams speak different native languages? …  Organization attributes matter. They can be observed, measured, and their impacts predicted.  If we do not explicitly predict, we assume that Slide 38 teams behave according to (our) past experience. Global Project Design © 2012
    39. 39. Case  Can one predict coordination and its Example impact on a program’s likely duration, cost, and risk?  If we can see impacts of complexity ahead of time, what might we do to: ̵ Reduce scope? ̵ Re-organize the system? ̵ Re-system the organization? Slide 39Global Project Design © 2012
    40. 40. Coordination • Coordination, Low Utilization, is Predicted & Rework Predictedin a Complex • Not only the amount of Industrial coordination, but when, where, Case and WHY it is demanded 40 Global Project Design © 2012
    41. 41. Resources: Teams Size, Location, Calendar, Abilities Architectural choices within a Architecture: Dependence, Roles, OBS, WBS & PBS Scenarios: complex program are a lever for better performance Externals: Targets, Start, Supplier Delivery… Basis of Scope: Activities, Direct Work, Complexity…Project Model 30 Change 25 20 15 10 # of scenarios 5 0 WKSP 1 day 1 WKSP 1 day 2 WKSP 1 day 3 WKSP 2 WKSP 3 WKSP 4 Workshop Session Global Project Design © 2012
    42. 42. Interfaces  Does an interface allow reduced or no interaction? In what horizon?  Is an interface a call to interact? Should a team pay more attention to the interface?  What happens when an interface breaks? Our teams need to be engaged, to interact and respect that which we yet don’t know or will discover Slide 42Global Project Design © 2012
    43. 43.  Normally work cultures evolve over time to Design of align behaviors, promote learning, and controlProjects in a risk. Complex ̵ Through a stable career one knew which interactionsEnvironment mattered, and others were “on the same page.” ̵ Today coordination and risk arise in unexpected places  Attention to coordination is not “soft”. These are real attributes of time, cost, and quality  Cases in complex global industrial programs confirm observed behaviors and sufficient predictability  Design of a project architecture can weigh team capacities and strengths, coordination, and flexibility Slide 43 Global Project Design © 2012
    44. 44. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?Looking at Complexity from the Outside InComplexity & Teams• Dialogue 44
    45. 45. Wrap Up 1. Organizations and the technical systems we engineer – are more and more complex 2. SE as framed today is strained and needs to respond 3. Changes are needed which include observation, analysis, and integration of socio-technical dimensions Slide 45Global Project Design © 2012
    46. 46. Dialogue  Does this hypothesis resonate with your experiences and practices?  Do you agree that SE needs to evolve  What tough questions should we be asking and researching about the SE process?  What are best practice examples in current programs which recognize of these real world behaviors? How was the SE process adapted? Slide 46Global Project Design © 2012