Emergent Architecture - March 2011
Upcoming SlideShare
Loading in...5
×
 

Emergent Architecture - March 2011

on

  • 3,183 views

Balancing Up-Front Planning and Emergence

Balancing Up-Front Planning and Emergence

Statistics

Views

Total Views
3,183
Slideshare-icon Views on SlideShare
2,899
Embed Views
284

Actions

Likes
0
Downloads
73
Comments
0

3 Embeds 284

http://www.leadingagile.com 282
http://twitter.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • What are the challenges with this description? There are two: 1) It assumes that EA is an ivory tower, and 2) it assumes that EA is about BUFP. Successful EA is neither of these things!
  • Evolutionary architecture and emergent design: Investigating architecture and designhttp://public.dhe.ibm.com/software/dw/java/j-eaed1-pdf.pdf
  • Why is it hard to accept? Because we’re wired for hierarchy. We’re trained accordingly. We live in conventional contexts. And we really, really, want to control.
  • Establish constraints: boundaries and goals. Because complexity science doesn’t tell you to simply wait for the rightsolutions to emerge. The way managers define boundaries andconstraints strongly influences what emerges from a self-organizing team[Lewin 2001]. You don’t manage the people. You manage the system.

Emergent Architecture - March 2011 Emergent Architecture - March 2011 Presentation Transcript

  • Enterprise Architecture
    Balancing Upfront Planning and Emergence
  • Published Description of this Talk
    “How do we handle the balance between the big up front plan, the ivory tower EA team, with the need to inspect and adapt and respond to the emerging needs of our customers?
    Can you identify the challenges with this description?
  • Survey Question #1What stakeholder groups are represented in the audience?
    End-Users
    The Business (sales, marketing, key management, etc)
    Customers
    Domain Experts (analyst, architect, systems engineer, etc)
    Developers (designer, coder, tester, etc)
  • Survey Question #2How many of you consider yourself an “agilist”?
    Agile Development
    Waterfall Development
    Iterative Development
    Iterative and Incremental Development
    Parallel Test Development
    Acceptance Test Driven Development
    Measure of Success
    Conformance to Plan
    Constant Flow of Business Value
    Process
    Handoffs/Sequential
    Teamwork/Parallel
    Culture
    Command and Control
    Leadership and Collaboration
    Design
    Big Design Up Front
    Continuous
    QA
    Big Test on Backend
    Continuous
    Tool Support
    Highly Specific
    Fully Integrated
    Lean and Efficient Value Streams
  • Which is Better?
    Predictive?
    Adaptive?
  • Which is Better?
    Trick Question
    Obviously Both Are Required
    Balance Is Required
    Balance is the Real Trick!
    Predictive
    Adaptive
  • Balance Is The Trick
    “Any development effort should be a balance between anticipation (planning based on what we know) and adaptation (responding to what we learn over time).”
    Jim Highsmith, Embracing Change, 17th March 2011
  • How Do We Find The Right Balance?
    “It Depends”
    “It’s Situationally Specific”
    “It’s All About The Context”
    Shucks! I Want It To Be Easy!
    Don’t you hate those answers?
  • How Do We Make It “Easier”?
    Focus On The Goal!
    Huh? What’s the Goal?
  • What Is Our Goal?
    Every Business Is Exactly The Same
    Every Business Has Exactly The Same Goal
    Every Business Is In Business…
    TO MAKE MONEY!
  • So, How Much Should We Plan?(How do we find the right balance?)
    Plan as much as necessary to receive a positive ROI on the planning investment
    We Plan Only To
    Maximize Value
    Delivery
  • Balance Is The Trick
    “If one has strong discipline without agility, the result is bureaucracy and stagnation. Agility without discipline is the unencumbered enthusiasm of a start-up company before it has to turn a profit.”
    Balancing Agility and Discipline, Barry Boehm and Richard Turner
  • Survey Question #3On A Scale of 1 – 5, How Important Is Planning?
    Planning Is The Purest Form Of Evil



    Planning Is The True Secret Sauce of Development
    Another Trick Question. Have You Been Paying Attention? 
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Survey Question #4How large are the companies represented by this audience?
    < 500 Associates
    501 – 1,000 Associates
    1,001 – 5,000 Associates
    5,001 – 25,000 Associates
    > 25,000 Associates
  • Definition of Architecture
    “Architecture is the fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution.” [IEEE 1471]
    IEEE Computer Society, IEEE Recommended Practice for Architectural Description of Software-Intensive Systems: IEEE Std 1472000. 2000.
  • Definition of Architecture
    “The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.”
    Bass, Len; Clements, Paul; & Kazman, Rick. Software Architecture in Practice, Second Edition. Boston, MA: Addison-Wesley, 2003.
  • Definition of Architecture
    Software Architecture encompasses the set of significant decisions about the organization of a software system:
    • Selection of the structural elements and their interfaces, by which a system is composed
    • Behavior as specified in collaborations among those elements
    • Composition of these structural and behavioral elements into larger subsystems
    • Architectural style that guides this organization
    G. Booch, P. Krutchen, K. Bittner and R. Reitman. The Rational Unified Process — An Introduction. 1999. Definition derived from Mary Shaw’s definition presented in 1995 at the First International Workshop on Architectures for Software Systems.
  • Definition Of Architecture
    "Architecture is about the important stuff. Whatever that is.“
    Who Needs An Architect?, Martin Fowler
    http://martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf
    Architecture is the stuff that's hard to change later. And there should be as little of that stuff as possible.
    Martin Fowler via Neil Ford
    Evolutionary architecture and emergent design: Evolutionary architecture
    http://public.dhe.ibm.com/software/dw/java/j-eaed10-pdf.pdf
  • Setting The Record Straight
    Architecture Has A Tremendous Impact On Results
    Few properties have as much impact on an organization’s success as does architecture.
    There is ALWAYS an Architecture
    Architecture is an inherent property of your organization and your software. You may not understand it. You may not communicate it. You may not have planned it. But it’s there!
    Architecture <> Documentation
    Architecture is that inherent property of the system – not the artifacts that describe it.
    Architecture <> Infrastructure / Hardware
    Architecture encompasses the myriad of perspectives and concerns that make up a solution - and enable its production. That includes hardware, software, operations, support, organization, etc.
    These Concepts Apply To Both “Enterprise” and “Application”
    For purposes of today’s discussion – how does emergence apply to architecture and planning – there is no difference between “Enterprise Architecture” and “Application Architecture”
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Why Do We Do Architecture?
    To capture the stakeholder perspectives that affect design
    To embrace change and to reduce the cost of solving problems
    To create a shared vision across the team and stakeholders
    To smooth the decision-making process
    - Lean Architecture, Jim Coplien and Gertrud Bjørnvig
    Focus On The Goal!
  • What Is An Architect?
    “Leads the development of the system's software architecture, which includes promoting and creating support for the key technical decisions that constrain the overall design and implementation for the project.”
    - Rational Unified Process
  • What Is An Architect?
    “An architect is a business-minded person who best understands how to leverage technology to maximize profitability.”
    - Paul Preiss, IASA (paraphrasing)
  • What Is An Architect?
    “A true software architect is one who is a domain expert, who knows how to apply the domain expertise to the design of a particular system, and who materially participates in implementation.”
    - Lean Architecture, Jim Coplien and Gertrud Bjørnvig
  • Competencies By Role
    A Perfect 10! 
  • Architect Focus Over The Life of a Project
    Discovery
    Invention
    Implementation
    Focus
    The Architect
    is accountable from
    “Concept to Cash”
    Project Lifecycle
    IBM – Brian Selic
  • Survey Question #5True of False: My Organization Has Individuals Specifically Assigned To The Architect Role.
  • Why Do We Do Architecture?
    “Architects” are not required, but these outcomes must be satisfied!
    To capture the stakeholder perspectives that affect design
    To embrace change and to reduce the cost of solving problems
    To create a shared vision across the team and stakeholders
    To smooth the decision-making process
    Lean Architecture, Jim Coplien and Gertrud Bjørnvig
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Survey Question #6True or False?
    Emergence is a principle that
    encourages us to defer decisions to the
    “Last Responsible Moment” and avoid
    “Big Up Front Planning” or
    “Big Up Front Design”
  • One Definition Of Emergence
    Emergence is what happens when the whole is smarter than the sum of its parts. It's what happens when you have a system of relatively simple-minded component parts… and they interact in relatively simple ways. And yet somehow out of all this interaction some higher level structure or intelligence appears, usually without any master planner calling the shots. These kinds of systems tend to evolve from the ground up.
    • http://www.oreillynet.com/lpt/a/1574
  • Another Definition Of Emergence
    "They are bottom-up systems, not top-down. They get their smarts from below. In a more technical language, they are complex adaptive systems that display emergent behavior. In these systems agents residing on one scale start producing behavior that lies one scale above them: ants create colonies; urbanites create neighborhoods; simple pattern-recognition software learns how to recommend new books. The movement from low-level rules to higher-level sophistication is what we call emergence.“
    - Emergence, Steven Johnson
  • Shorter Definition Of Emergence
    Collective Intelligence
  • Stop The Insanity
    Emergence is notabout the timing of decisions.
    Emergence is about harnessing the incredible intelligence that lies dormant in our organizations – trapped in antiquated and outdated philosophies of management.
    Emergence is about enabling and empowering the people to achieve a greater purpose.
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Simple Principles Of Emergence
    The source of emergence is the interaction among agents who mutually affect each other.
    Attend to relationships characterized by mutuality among people, among teams and among companies in order for novelty to emerge.
    Small change can lead to large effects.
    Seek to lead change through many small experiments, which search the landscape of possibilities.
    Emergence is certain, but there is no certainty of what it will be.
    Create conditions for constructive emergence rather than to plan a strategic goal in detail. Evolve solutions, don’t design them.
    Greater diversity of agents in a system leads to richer emergent patterns.
    Seek a diversity of people, cultures, expertise, ages, personalities, gender so that when they interact in teams, creativity has the potential of being enhanced.
    - Complexity: life at the edge of chaos, Roger Lewin
    Simple to Describe. Hard to Accept!
  • How Do We Foster Emergence?
    Accept that there is in fact a “Complex Adaptive System”
    Provide leadership in the design of the system
    Manage the system – not the people
    Quit thinking linearly 
    Passionately apply a “new management model”
  • A New Model For Management
    Management Needs Changing
    New model leveraging “Complexity Theory”
    Acknowledge that organizations are really networks
    6 Important Practices (“views”)
    Energize People
    Empower Teams
    Align Constraints
    Develop Competence
    Grow Structure
    Improve Everything
    - Management 3.0, JurgenApello
  • View #1: Energize People
    People are the most important parts of an organization and managers must do all they can to keep people active, creative, and motivated.
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • View #2: Empower Teams
    Teams can self-organize, and this requires empowerment, authorization, and trust from management.
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • View #3: Align Constraints
    Self-organization can lead to anything, and it’s therefore necessary to protect people and shared resources, and to give people a clear purpose and defined goals.
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • View #4: Develop Competence
    Teams cannot achieve these goals if team members aren’t capable enough, and managers must therefore contribute to the development of competence.
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • View #5: Grow Structure
    Many teams operate within the context of a complex organization, and thus it is important to consider structures that enhance communication .
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • View #6: Improve Everything
    People, teams, and organizations need to improve continuously to defer failure for as long as possible.
    • Complexity Versus Lean, JurgenApello
    • “http://www.slideshare.net/jurgenappelo/complexity-versus-lean
  • “Nurture” Emergence
    Passionately Apply a New
    Management Model That
    1. Establishes Clear Context, and
    2. Energizes and Empowers The People
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Recipe For Value Delivery
    Create The Opportunity
    Design The “System”
    Nurture Emergence
    Apply A New Management Model
    Execution Loop
    Do Just Enough Planning
    Create The Context
    Establish The Guard Rails
    Lubricate Execution
    Get On With It
    Sense and Respond
    Repeat
  • What Planning Do We Need To Do?
    Plan the right things at the right time
    Just enough to “accomplish the goal” by:
    Establishing Clear Context
    Energizes and Empowering The People
    Typically Focus On These Outcomes:
    Capture the stakeholder perspectives that affect the priority aspects of the design
    Embrace change and to reduce the cost of solving problems
    Create a shared vision across the team and stakeholders
    Smooth the decision-making process
  • Emergence and Decisions
    Making decisions sets the context required to enable emergence
    There are often multiple “scales” involved
    Decision cycles are different at different scales
    Every scale can operate in an emergent fashion
    Strategy/Vision/Mission -> Enterprise -> Application
    Vertical feedback loops are required
  • How Much is “Enough”?
    “In these systems agents residing on one scale start producing behavior that lies one scale above them.”
    - Emergence, Steven Johnson
  • Establishing Context
    Context for Energizing and Empowering
    Context for Safety
    Context for Efficiency
  • Context For Energizing / Empowering
    A compelling description of the goals
    Sense of urgency and purpose
    Alignment with vision, mission, and strategy
  • Empowerment
    “If you want to build a ship, don't drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.”
    - Antoine de Saint Exupéry
  • Emergence and Uncertainty
    Emergence involves some degree of uncertainty
    Empowerment introduces independence
    Does the Heisenberg Uncertainty Principle play?
  • Emergence and Leadership
    "Good leaders create the conditions for constructive emergence rather than trying to plan every strategic goal in detail.“
    • Step Back & Emerge, Dr Paul Thomas
    • http://businessdoctorme.blogspot.com/2010/07/step-back-emerge.html
  • Context For Safety
    Clarity of enterprise strategy and objectives
    “Good enough” description of the enterprise target architecture
    Essential architectural requirements and constraints
  • Emergence and Boundaries
    Boundaries Establish Freedom
    Is This contrary to popular belief?
    Freedom is Required For Speed and Agility
  • Context for Efficiency
    Not all decisions are created equal
    Some things are harder to change than others
    Some decisions are more “core” or “fundamental” and get the ball rolling
    Don’t forget “strategic” vs “tactical”
    Evolutionary architecture and emergent design: Evolutionary architecture
    http://public.dhe.ibm.com/software/dw/java/j-eaed10-pdf.pdf
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
  • Can We Let Go?
    Understanding emergence is about: giving up control; being more tolerant of that exploratory phase where the rules don't make sense and where few goals have been clearly defined; letting the system govern itself as much as possible; and letting it learn from its own footprints.
    • http://www.cleanlanguage.co.uk/articles/articles/216/1/Self-Organising-Systems-Findhorn/Page1.html
  • Nurture The Freak
    a
    r
    F
    k
    e
  • The World Is A Fractal
    “Once you start thinking of strategy as an emergent phenomenon, you realize that we have often attacked the wrong end of the problem. Strategists and senior executives have too often worked on "the strategy," rather than on the preconditions that could give rise to strategy innovation. In essence, they've been trying to design complex, multicell organisms, rather than trying to understand and create the conditions from which such organisms will emerge.”
    - “Strategy Innovation and the Quest for Value, Gary Hamel (http://www.strategos.com/articles/questforvalue.htm)
  • Q&A
  • Agenda
    Introduction
    What is Architecture?
    What is an Architect?
    What is Emergence?
    How Can We Foster Emergence?
    How To Maximize Value Delivery
    Conclusion and Q&A
    Appendix (Gartner On Emergent Architecture)
  • Gartners “New” Emergent EA
    “Enterprise architects must adopt a new style of enterprise architecture (EA) to respond to the growing variety and complexity in markets, economies, nations, networks and companies, according to Gartner, Inc.  Analysts advised companies to adopt ‘emergent architecture’, also known as middle-out EA and light EA, and set out definitions of the new approach.”
    Key Characteristics of the Emergent Approach
    “Summarised as ‘architect the lines, not the boxes’, which means managing the connections between different parts of the business rather than the actual parts of the business themselves.”
    “Models all relationships as interactions via some set of interfaces, which can be completely informal and manual – for example, sending handwritten invitations to a party via postal letters - to highly formal and automated, such as credit-card transactions across the Visa network.”
    Bruce Robertson, research vice president at Gartner
    Gartner Press Release, Egham, UK, August 11, 2009
    http://www.gartner.com/it/page.jsp?id=1124112
  • Gartner Emergent EA Principles#1 Non-Deterministic
    Non-deterministic - In the past, enterprise architects applied centralized decision-making to design outcomes. Using emergent architecture, they instead must decentralize decision-making to enable innovation.
    I've heard about this "deterministic" EA practice. And I've also heard about unicorns. Every effective EA practice I've seen recognized its role as one of leadership - and context. One of the biggest drivers in business today is Agility - the need to respond rapidly to changing needs and opportunities. By definition, then, we are operating in a climate where the future is not pre-determined or predicted. As such, at some scale, the specific outcomes are obviously non-deterministic. But there's a huge risk of this property being abused (see Agile Is Not "Make It Up As You Go"). As a whole, any organization with an EA practice absolutely has some destination in mind... some target. It's our job to create context and to provide leadership that helps the organization translate that target into actionable goals... and to adapt its way to success.
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#2 Autonomous Actors
    Autonomous actors - Enterprise architects can no longer control all aspects of architecture as they once did. They must now recognise the broader business ecosystem and devolve control to constituents.
    Again, the idea that an Enterprise Architect could ever "control all aspects of architecture" is a farce. The power of an organization always lies with those that are serving the organizations clients - the business units. Our role in EA is to serve those people on the front line and empower them to better meet the needs of their customers - and at the same time advance the organization as a whole towards its targets. This is a role of leadership - not control - and I wrote about it in Leadership - The Secret Sauce. Emphasizing the value of "collective intelligence," that post reminds us that we can "achieve outrageous levels of performance by harnessing the intellect and energy of the people." This also came up in Nurture The Freaks where we contemplated these words from Gary Hamel, "Going forward, no company will be able to afford to waste a single iota of human imagination and intellectual power."
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#3 Rule-Bound Actors
    Rule-bound actors - Where in the past enterprise architects provided detailed design specifications for all aspects of the EA, they must now define a minimal set of rules and enable choice.
    It's a reasonably well accepted principle that an EA practice should never make a decision (or set a constraint) that could be left to the business unit or development team. In fact, enabling choice and encouraging participation are important vehicles for gaining buy-in and goodwill (see Governance Without Goodwill Is Dead). This is yet another reminder of how we need to establish context by creating guard rails that keep the organization out of the ditch.
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#4 Goal-Oriented Actors
    Goal-oriented actors - Previously, the only goals that mattered were the corporate goals but this has now shifted to each constituent acting in their own best interests.
    With the guard rails in place, responsibility for driving rests on the individual drivers, each in their own vehicle with both hands on the wheel.
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#5 Local Influences
    Local Influences - Actors are influenced by local interactions and limited information. Feedback within their sphere of communication alters the behaviour of individuals. No individual actor has data about all of an emergent system. EA must increasingly coordinate.
    There is a massive amount of information flowing through the modern organization, and the majority of it originates and circulates right on the front line where the dynamic nature of today's agile organization demands a high degree of "in the heat of battle" decision-making. This suggests EA add value by encouraging a broad community that is willing and able to actively contribute to and consume a real-time, high-bandwidth stream of communication. It's not our job to assimilate it all and make decisions. Instead, as individual consumers at the trough of the information stream, we help drive the information out to those who need it the most. I don't really like the use of the word "coordinate" here. We're only coordinating in the indirect sense. Perhaps this role is some combination of the Connector, Maven, and Salesman roles described by Malcolm Gladwell in The Tipping Point.
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#6 Dynamic or Adaptive Systems
    Dynamic or Adaptive Systems - The system (the individual actors as well as the environment) changes over time. EA must design emergent systems [that] sense and respond to changes in their environment.
    This is one of the most important functions of the Enterprise Architecture discipline in a modern organization. We have a responsibility to bring a "systems thinking" perspective to the table and influence the design of flexible and adaptive systems - systems that have the ability to learn from and respond to their experience. When talking about systems, here, it's critical that we deliberately design this adaptive nature into our products AND our organizations. I'm happy to see that Gartner is beginning to recognize organizations as a type of "complex adaptive system".
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html
  • Gartner Emergent EA Principles#7 Resource-Constrained Environment
    Resource-Constrained Environment - An environment of abundance does not enable emergence; rather, the scarcity of resources drives emergence.
    Most of us are operating under the influence of unprecedented economic conditions, and these times demand that we become more creative. In fact, creative isn't really the right word. To respond to the reality of the corporate climate of today (and tomorrow), we're going to need to be "clever" - adroit, nimble, resourceful, and mentally quick. The organizations that thrive in the future will be those that respond today by building a sustainable system of Agile capabilities that maximize the contribution of every associate.
    http://blog.softwarearchitecture.com/2009/09/effective-enterprise-architecture.html