P.W.Anderson states in his classic paper titled "More is Different" - The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe. He further states - The constructionist hypothesis breaks down when confronted with the twin difficulties of scale and complexity. The behavior of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviors requires research (fundamental). Complex systems are characterized by: Extraordinary decentralization Inherently conflicting, unknowable and diverse requirements Continuous evolution and deployment Heterogeneous, inconsistent, and changing elements Erosion of people/system boundary Normal failures New paradigms for acquisition and policy Thus far our methods to confront complexity have been based on reductionism or analysis, determinism, dualism, correspondence theory of knowledge and rationality – analytical and logical thinking as we know it. They have worked well for us in the past and continue to drive our approaches to problem solving, change creation and innovation. However, the new age of innovation warrants newer methods to deal with complexity. These new methods are likely to be based on a deeper understanding of indeterminacy, non-linearity, chaos, adaptation, self-organization and distributed intelligence. Crafitti provides an integrated approach to Ultra-large scale systems design using the Lean Inventive Systems Thinking Framework. This was presented at the DesignFirst Conference 2008 held at Bangalore.

1. crafting innovation together
Designing Ultra Large Scale Systems
- The LIST* Approach
crafting innovation together
Navneet Bhushan & Karthikeyan Iyer
Crafitti Consulting
Oct 17, 2008
* Lean Inventive Systems Thinking

2. More is Different – Scale is the New Frontier - I
crafting innovation together
Google’s Custom Built Server
Farms
Current estimates put Google's server
farm at around 450,000 machines - and
they're still custom built, commodity-
class x86 PCs, just like they were in 1999
crafting innovation together
We Are Building Bigger
and Bigger Systems
Oct 17, 2008

3. More is Different – Scale is the New Frontier - II
crafting innovation together
The Internet Inter-disciplinary Capillary network
collaborations
crafting innovation together
Complex system design
diagram
High School
Friendships
Systems are Evolving into Bigger Systems
Oct 17, 2008

4. More is Different – Scale is the New Frontier - III
crafting innovation together
“The ability to reduce everything to simple fundamental
laws does not imply the ability to start from those laws
and reconstruct the universe” Anderson, P.W., More is Different,
Science, Vol. 177, No. 4047, Aug. 4, 1972, pp. 393-396.
crafting innovation together
Scale Changes Everything!
Oct 17, 2008

5. More is Different – Scale is the New Frontier - IV
crafting innovation together
“The older is not always a reliable model for the newer, the
smaller for the larger, or the simpler for the more
complex…Making something greater than any existing thing
necessarily involves going beyond experience.”
Henry Petroski, Pushing the Limits: New Adventures in Engineering
crafting innovation together
Scale is not a linear extrapolation!
Oct 17, 2008

6. SEI ULS Study – 2006!
crafting innovation together
crafting innovation together
Oct 17, 2008

7. Why we don’t know how to Design ULSS?
crafting innovation together
System Design - Present Approaches ULS Characteristics
All conflicts must be resolved centrally and uniformly Decentralized Control
Requirements can be known in advance and change slowly. Inherently conflicting, unknowable,
Tradeoff decisions will be stable. and diverse requirements
Discrete Time System improvements Continuous evolution and
Effect of a change can be predicted sufficiently well.
Configuration information
Components and users are fairly homogeneous.
People are just users of the system.
? deployment
Heterogeneous, inconsistent,
changing elements
and
crafting innovation together
Erosion of the people/system
Collective behavior of people is not of interest. boundary
Social interactions are not relevant.
Normal Failures
Failures will occur infrequently. Defects can be removed.
A prime contractor is responsible for system development,
New paradigms for acquisition and
operation, and evolution. policy
Oct 17, 2008

8. ULSS Research Areas
crafting innovation together
ULS Systems Specific Sub-Areas
Research Area
• Decentralized Production Management
• Context-Aware Assistive computing Adaptive • View-Based Evolution
• Understanding Users and Their Contexts
• Modeling Users and User Communities
System • Evolutionary Configuration and
Human Interaction • Fostering Non-Competitive Social Infrastructure Deployment
• In Situ Control and Adaptation
Collaboration
• Robustness, Adaptation, and Quality
• Longevity
Attributes
• Algorithmic Mechanism Design
Computational • Scale and Composition of Quality
• Metaheuristics in Software Engineering
Emergence Adaptable and Attributes
• Digital Evolution
• Understanding People-Centric Quality
• Design of All Levels Predictable
crafting innovation together
Attributes
• Design Spaces and Design rules
System Quality • Enforcing Quality Requirements
• Harnessing Economics to Promote Good
• Security, Trust, and Resiliency
Design
Design • Design Representation and Analysis
• Engineering Management at Ultra-
Large Scales
• Assimilation
• Determining and Managing Policy,
• Policy Definition for ULS Systems
Requirements Acquisition,
• Expressive Representation Languages • Fast Acquisition for ULS Systems
and • Management of ULS Systems
• Scaled-Up Specification, Verification, and
Computational Management
Certification
Engineering • Computational Engineering for Analysis
and Design
Oct 17, 2008

9. crafting innovation together
LEAN INVENTIVE SYSTEMS
THINKING (LIST)
CLASSICAL THE LIST
REDUCTIONISM
crafting innovation together
Learning, Discovery, Design,
Analysis, Determinism, Evolutionary, Experimental,
Dualism, Correspondence Integrative, Holistic, Non-
theory of knowledge, linear, Natural
Rationality, Artificial
Oct 17, 2008

10. Elements of System Design
crafting innovation together
Needs
Designing ULS
Behavior Function
Systems
crafting innovation together
Structure
Oct 17, 2008

11. Designing for Complex Needs
crafting innovation together
Sight
Intra-… 4 Sound
3
2 Smell
1
Kinesthetic 0 Taste
Spatial Touch
Logical Linguistic
Musical
crafting innovation together
Oct 17, 2008

12. Powerful Functional Design – TRIZ approach
crafting innovation together
Improving
parameter
Worsening
parameter
• Eliminate System
Contradictions
Increasing dynamism (flexibility)
Transition to higher level systems
Multiple design Transition to micro level systems
alternatives and • Move to a higher level Completeness (reducing human involvement)
along proven lines of
paths (Which one to Shortening of energy flow path
crafting innovation together
system evolution Increasing controllability
choose?)
Harmonization of rhythms
Non-uniform evolution of sub-systems
• Ideal Final Result
Benefits
Ideality Quotient =
(Cost + Harmful Effects)
TRIZ: Theory of Inventive Problem Solving
Oct 17, 2008

13. Structure – Complexity and Centrality
crafting innovation together
Why are structures rigid?
Stability - They are elements where the most critical
functions are performed and cannot afford to fail
Unpredictability – They are not very well
understood and are unpredictable, hence they are
deliberately operating within strict constraints
Dependency – Too many other system elements are
crafting innovation together
dependent on this element, hence it cannot be
changed very easily and without pain
Insulation – They are not very well connected and
Flexibility, Adaptability, therefore do not have an incentive to change or
Ability to evolve, change adapt to changes
Efficiency – The elements are optimally structured
to perform certain functions as efficiently as
Stability, Rigidity, Strength, possible
Productivity, Efficiency
Oct 17, 2008

14. System Behavior
crafting innovation together
• Behavior of the “whole” as opposed to the “parts”
• Need for observation from a different plane
• Complex non-linear systems display macro behavior invisible when
seen from inside the system
– Synchronization
– Chaos
crafting innovation together
– Balance
Oct 17, 2008

15. System Behavior –The tendency to synchronize
crafting innovation together
• Synchronization aids stability –
Arrhythmia and cessation
of breath centralized control difficult in
complex non-linear systems
• Synchronization is learnt over time;
complex non-linear systems need to
be designed as “learning systems”
Toy rhythm
• Systemic synchronization is a result
of breath of distributed intelligence
crafting innovation together
• Synchronization follows simple rules
at the sub-system level
• Synchronization happens around
system rhythms or clocks
• Synchronization happens for a
Child rhythm reason (beneficial outcome)
of breath
Oct 17, 2008

16. System Behavior - Designing for Chaos
crafting innovation together
• Chaotic systems hide certain
patterns of behavior called
attractors.
• Complex non-linear systems display
chaotic behavior and gravitate
crafting innovation together
towards system attractors
• Attractors manifest across scale
• Attractors act as strong central
rhythms or clocks.
• Strange Attractors
Oct 17, 2008

17. Design – A Macro“Balancing” act
crafting innovation together
Fire
Air
Fuel, Energy to run the
Distribution, change
system
Complex ULS
System
crafting innovation together
Water Earth
Vitality, Life, Growth - Ideas Structure and raw material
Oct 17, 2008

18. The “Typical”Design Process
crafting innovation together
Always Reaching Local Optima
Creates a false sense of
simplification of complexity
crafting innovation together
Introduces artificial delay
Oct 17, 2008

19. The Lean Design Approach –
Set-Based Concurrent Engineering
crafting innovation together
 Search for global optima
 Elimination of the weakest
 Slow convergence
 Emergent design
 Scope to incorporate systems
crafting innovation together
thinking (needs, functions,
structure and behavior) and
inventive thinking (TRIZ)
Oct 17, 2008

20. The Design Process and the LIST framework
crafting innovation together
Mapping the  Describe user needs • Problem Formulation and Analysis
 In case of multiple needs carry out needs • Value Stream
Design Space
interdependency analysis • Ideal Final Result (IFR)
• Why-what hierarchy
 Find out key functions to be performed
• Nine windows
 Understand structural complexities • Dependency Structure Matrix (DSM)
 Understand behavioral complexities • Function/Attribute Analysis
 Function dependency analysis to find out • System Complexity Estimator (SCE)
interdependencies • S curve analysis
 Can some high level functions specific to • Vedic Inventive Principles
• Contradictions – Technical/Physical
strengths of different teams be identified
crafting innovation together
• Trends of evolution
 Let each team explore the specifications,
needs, functions independent of each
other
 Each team explore design tradeoffs
through simulations and their past
observations
 Each team should come up with their sets
of different solutions within the functional
and performance needs of the product
Oct 17, 2008

21. Design Process and LIST continued . . .
crafting innovation together
Striving for Conceptual  Design should remain functional  IFR
Robustness after variations in its  AFD/Subversion Analysis
environment  Robust Inventive System Design (RISD)
(Functional Team level)
 Vulnerability of system to  DSM
changes in the environment
should be minimized
 Modularized Design with
standard components
Integration by  How are the parts integrated to  Decision Dependency Matrices (DDM)
crafting innovation together
Intersection meet at the point that will be  Analytic Hierarchy Process (AHP)
regarded best solution  Technical Contradictions / Inventive
(System level)
 Find out overlap of feasible Principles
design spaces for each sub
component
 Decisions about eliminating the
weak designs
Oct 17, 2008

22. Design Process and LIST continued . . .
crafting innovation together
Establish Feasibility  Multiple concepts developed using  Decision theoretic principles
before prototyping simulation  AHP
 The infeasible ones will be rejected rest all  Closer to IFR
Commitment
will continue to be developed
Conflict Handling Cooperative Conflict handling  Which solution is closer to IFR?
 DDM
 AHP
crafting innovation together
Oct 17, 2008

23. Final Points
crafting innovation together
• Scale is the New Frontier – it changes everything!
• Existing approaches found insufficient for designing Ultra Large
Scale Systems
• Need for designing learning, discovery, human-machine
cohesiveness and failure absorption inside the ULSS
• We propose a new framework combining elements of ancient
wisdom, modern complexity science, empirical theory of
crafting innovation together
invention, practical experiences of striving excellence and holistic
design principles
• Lean Inventive Systems Thinking (LIST) may help us to bridge
the gap between current approaches and needs of ULSS
Oct 17, 2008

24. THANK YOU!
crafting innovation together
Lean Inventive Systems Thinking
Crafitti Consulting
Crafting innovation together . . .
www.crafitti.com
crafting innovation together
Navneet Bhushan (navneet.bhushan@crafitti.com)
Karthikeyan Iyer (karthikeyan.iyer@crafitti.com)
Oct 17, 2008