Designing Ultra Large Scale Systems List

Uploaded on

P.W.Anderson states in his classic paper titled "More is Different" - The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct …

P.W.Anderson states in his classic paper titled "More is Different" - The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe. He further states - The constructionist hypothesis breaks down when confronted with the twin difficulties of scale and complexity. The behavior of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviors requires research (fundamental).

Complex systems are characterized by:

Extraordinary decentralization
Inherently conflicting, unknowable and diverse requirements
Continuous evolution and deployment
Heterogeneous, inconsistent, and changing elements
Erosion of people/system boundary
Normal failures
New paradigms for acquisition and policy

Thus far our methods to confront complexity have been based on reductionism or analysis, determinism, dualism, correspondence theory of knowledge and rationality – analytical and logical thinking as we know it. They have worked well for us in the past and continue to drive our approaches to problem solving, change creation and innovation.

However, the new age of innovation warrants newer methods to deal with complexity. These new methods are likely to be based on a deeper understanding of indeterminacy, non-linearity, chaos, adaptation, self-organization and distributed intelligence.

Crafitti provides an integrated approach to Ultra-large scale systems design using the Lean Inventive Systems Thinking Framework.

This was presented at the DesignFirst Conference 2008 held at Bangalore.

More in: Design , Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. crafting innovation together Designing Ultra Large Scale Systems - The LIST* Approach crafting innovation together Navneet Bhushan & Karthikeyan Iyer Crafitti Consulting Oct 17, 2008 * Lean Inventive Systems Thinking
  • 2. More is Different – Scale is the New Frontier - I crafting innovation together Google’s Custom Built Server Farms Current estimates put Google's server farm at around 450,000 machines - and they're still custom built, commodity- class x86 PCs, just like they were in 1999 crafting innovation together We Are Building Bigger and Bigger Systems Oct 17, 2008
  • 3. More is Different – Scale is the New Frontier - II crafting innovation together The Internet Inter-disciplinary Capillary network collaborations crafting innovation together Complex system design diagram High School Friendships Systems are Evolving into Bigger Systems Oct 17, 2008
  • 4. More is Different – Scale is the New Frontier - III crafting innovation together “The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe” Anderson, P.W., More is Different, Science, Vol. 177, No. 4047, Aug. 4, 1972, pp. 393-396. crafting innovation together Scale Changes Everything! Oct 17, 2008
  • 5. More is Different – Scale is the New Frontier - IV crafting innovation together “The older is not always a reliable model for the newer, the smaller for the larger, or the simpler for the more complex…Making something greater than any existing thing necessarily involves going beyond experience.” Henry Petroski, Pushing the Limits: New Adventures in Engineering crafting innovation together Scale is not a linear extrapolation! Oct 17, 2008
  • 6. SEI ULS Study – 2006! crafting innovation together crafting innovation together Oct 17, 2008
  • 7. Why we don’t know how to Design ULSS? crafting innovation together System Design - Present Approaches ULS Characteristics All conflicts must be resolved centrally and uniformly Decentralized Control Requirements can be known in advance and change slowly. Inherently conflicting, unknowable, Tradeoff decisions will be stable. and diverse requirements Discrete Time System improvements Continuous evolution and Effect of a change can be predicted sufficiently well. Configuration information Components and users are fairly homogeneous. People are just users of the system. ? deployment Heterogeneous, inconsistent, changing elements and crafting innovation together Erosion of the people/system Collective behavior of people is not of interest. boundary Social interactions are not relevant. Normal Failures Failures will occur infrequently. Defects can be removed. A prime contractor is responsible for system development, New paradigms for acquisition and operation, and evolution. policy Oct 17, 2008
  • 8. ULSS Research Areas crafting innovation together ULS Systems Specific Sub-Areas Research Area • Decentralized Production Management • Context-Aware Assistive computing Adaptive • View-Based Evolution • Understanding Users and Their Contexts • Modeling Users and User Communities System • Evolutionary Configuration and Human Interaction • Fostering Non-Competitive Social Infrastructure Deployment • In Situ Control and Adaptation Collaboration • Robustness, Adaptation, and Quality • Longevity Attributes • Algorithmic Mechanism Design Computational • Scale and Composition of Quality • Metaheuristics in Software Engineering Emergence Adaptable and Attributes • Digital Evolution • Understanding People-Centric Quality • Design of All Levels Predictable crafting innovation together Attributes • Design Spaces and Design rules System Quality • Enforcing Quality Requirements • Harnessing Economics to Promote Good • Security, Trust, and Resiliency Design Design • Design Representation and Analysis • Engineering Management at Ultra- Large Scales • Assimilation • Determining and Managing Policy, • Policy Definition for ULS Systems Requirements Acquisition, • Expressive Representation Languages • Fast Acquisition for ULS Systems and • Management of ULS Systems • Scaled-Up Specification, Verification, and Computational Management Certification Engineering • Computational Engineering for Analysis and Design Oct 17, 2008
  • 9. crafting innovation together LEAN INVENTIVE SYSTEMS THINKING (LIST) CLASSICAL THE LIST REDUCTIONISM crafting innovation together Learning, Discovery, Design, Analysis, Determinism, Evolutionary, Experimental, Dualism, Correspondence Integrative, Holistic, Non- theory of knowledge, linear, Natural Rationality, Artificial Oct 17, 2008
  • 10. Elements of System Design crafting innovation together Needs Designing ULS Behavior Function Systems crafting innovation together Structure Oct 17, 2008
  • 11. Designing for Complex Needs crafting innovation together Sight Intra-… 4 Sound 3 2 Smell 1 Kinesthetic 0 Taste Spatial Touch Logical Linguistic Musical crafting innovation together Oct 17, 2008
  • 12. Powerful Functional Design – TRIZ approach crafting innovation together Improving parameter Worsening parameter • Eliminate System Contradictions Increasing dynamism (flexibility) Transition to higher level systems Multiple design Transition to micro level systems alternatives and • Move to a higher level Completeness (reducing human involvement) along proven lines of paths (Which one to Shortening of energy flow path crafting innovation together system evolution Increasing controllability choose?) Harmonization of rhythms Non-uniform evolution of sub-systems • Ideal Final Result Benefits Ideality Quotient = (Cost + Harmful Effects) TRIZ: Theory of Inventive Problem Solving Oct 17, 2008
  • 13. Structure – Complexity and Centrality crafting innovation together Why are structures rigid? Stability - They are elements where the most critical functions are performed and cannot afford to fail Unpredictability – They are not very well understood and are unpredictable, hence they are deliberately operating within strict constraints Dependency – Too many other system elements are crafting innovation together dependent on this element, hence it cannot be changed very easily and without pain Insulation – They are not very well connected and Flexibility, Adaptability, therefore do not have an incentive to change or Ability to evolve, change adapt to changes Efficiency – The elements are optimally structured to perform certain functions as efficiently as Stability, Rigidity, Strength, possible Productivity, Efficiency Oct 17, 2008
  • 14. System Behavior crafting innovation together • Behavior of the “whole” as opposed to the “parts” • Need for observation from a different plane • Complex non-linear systems display macro behavior invisible when seen from inside the system – Synchronization – Chaos crafting innovation together – Balance Oct 17, 2008
  • 15. System Behavior –The tendency to synchronize crafting innovation together • Synchronization aids stability – Arrhythmia and cessation of breath centralized control difficult in complex non-linear systems • Synchronization is learnt over time; complex non-linear systems need to be designed as “learning systems” Toy rhythm • Systemic synchronization is a result of breath of distributed intelligence crafting innovation together • Synchronization follows simple rules at the sub-system level • Synchronization happens around system rhythms or clocks • Synchronization happens for a Child rhythm reason (beneficial outcome) of breath Oct 17, 2008
  • 16. System Behavior - Designing for Chaos crafting innovation together • Chaotic systems hide certain patterns of behavior called attractors. • Complex non-linear systems display chaotic behavior and gravitate crafting innovation together towards system attractors • Attractors manifest across scale • Attractors act as strong central rhythms or clocks. • Strange Attractors Oct 17, 2008
  • 17. Design – A Macro“Balancing” act crafting innovation together Fire Air Fuel, Energy to run the Distribution, change system Complex ULS System crafting innovation together Water Earth Vitality, Life, Growth - Ideas Structure and raw material Oct 17, 2008
  • 18. The “Typical”Design Process crafting innovation together Always Reaching Local Optima Creates a false sense of simplification of complexity crafting innovation together Introduces artificial delay Oct 17, 2008
  • 19. The Lean Design Approach – Set-Based Concurrent Engineering crafting innovation together  Search for global optima  Elimination of the weakest  Slow convergence  Emergent design  Scope to incorporate systems crafting innovation together thinking (needs, functions, structure and behavior) and inventive thinking (TRIZ) Oct 17, 2008
  • 20. The Design Process and the LIST framework crafting innovation together Mapping the  Describe user needs • Problem Formulation and Analysis  In case of multiple needs carry out needs • Value Stream Design Space interdependency analysis • Ideal Final Result (IFR) • Why-what hierarchy  Find out key functions to be performed • Nine windows  Understand structural complexities • Dependency Structure Matrix (DSM)  Understand behavioral complexities • Function/Attribute Analysis  Function dependency analysis to find out • System Complexity Estimator (SCE) interdependencies • S curve analysis  Can some high level functions specific to • Vedic Inventive Principles • Contradictions – Technical/Physical strengths of different teams be identified crafting innovation together • Trends of evolution  Let each team explore the specifications, needs, functions independent of each other  Each team explore design tradeoffs through simulations and their past observations  Each team should come up with their sets of different solutions within the functional and performance needs of the product Oct 17, 2008
  • 21. Design Process and LIST continued . . . crafting innovation together Striving for Conceptual  Design should remain functional  IFR Robustness after variations in its  AFD/Subversion Analysis environment  Robust Inventive System Design (RISD) (Functional Team level)  Vulnerability of system to  DSM changes in the environment should be minimized  Modularized Design with standard components Integration by  How are the parts integrated to  Decision Dependency Matrices (DDM) crafting innovation together Intersection meet at the point that will be  Analytic Hierarchy Process (AHP) regarded best solution  Technical Contradictions / Inventive (System level)  Find out overlap of feasible Principles design spaces for each sub component  Decisions about eliminating the weak designs Oct 17, 2008
  • 22. Design Process and LIST continued . . . crafting innovation together Establish Feasibility  Multiple concepts developed using  Decision theoretic principles before prototyping simulation  AHP  The infeasible ones will be rejected rest all  Closer to IFR Commitment will continue to be developed Conflict Handling Cooperative Conflict handling  Which solution is closer to IFR?  DDM  AHP crafting innovation together Oct 17, 2008
  • 23. Final Points crafting innovation together • Scale is the New Frontier – it changes everything! • Existing approaches found insufficient for designing Ultra Large Scale Systems • Need for designing learning, discovery, human-machine cohesiveness and failure absorption inside the ULSS • We propose a new framework combining elements of ancient wisdom, modern complexity science, empirical theory of crafting innovation together invention, practical experiences of striving excellence and holistic design principles • Lean Inventive Systems Thinking (LIST) may help us to bridge the gap between current approaches and needs of ULSS Oct 17, 2008
  • 24. THANK YOU! crafting innovation together Lean Inventive Systems Thinking Crafitti Consulting Crafting innovation together . . . crafting innovation together Navneet Bhushan ( Karthikeyan Iyer ( Oct 17, 2008