Agile Architecture Made Real
Re-architecting the organization
for speed - Building a fast and
flexible architecture across 50
scrum teams
Introductions and why we are here
Alexis Hui
 Co-lead the Deloitte LEAN practice – Agile Advisory and
Transformation
 10+ years of agile experience - doing, advising and coaching
 Helping organizations make agile work @ scale (100’s-1000’s)
My goal is to share with you how agile can work at speed and
scale based on a case study to make it real
Who needs
architecture?
Scrum
XP
Kanban
Agile Modeling
Method
100’s 1000’s
# people
Ok… something is
missing
My toolkit worked great to help organizations become more
agile… until we started hitting larger organizations
Architecture is at the heart of making agile work @ scale
Move Fast and
Break Things
Self-organizing
teams
EXPERIMENT.
FAIL.
LEARN.
REPEAT.
Speed and scale – Oxymoron?
Large platform
 3M+ accounts
 $2B+ funds
 5.5M+ transactions / mth
 200+ product features with
multiple products
Distributed organization
 1000+ people
 60-70% outsourced
 7+ locations, 5 major
delivery centers
 5 time zones
Lots of Partners
 Co-branding
 FSI’s
 Enterprise back-ends
 Enterprise servicing
New Speed Demands
 Handle constantly changing
programs and business
development contracts
 Heavily regulated industry
with frequent changes in
regulations
 Highly competitive space
with “fast” competitors
 Enormous pressure to drive
big revenue growth and
acquire customers
 Desire to deploy and
release every two weeks
New business unit spun up 4 years ago to drive new revenue streams for a
much larger traditional parent FSI organization…
Rethinking the architecture – not just software
What is the real product?
 Reframing the world, the
product is the platform
How do we scale our architecture
to the next level?
 Organize teams around the
product architecture and let
teams do the work for you
How do we release faster than
ever?
 Re-architect everything we
have to release every two
weeks
…organizations which design systems ... are constrained to produce designs
which are copies of the communication structures of these organizations
– M. Conway
Agile Architecture in Action
Refactored their platforms with componentization in mind
• Platform refactored into a discrete set of products and platform services
• Agile is enabling teams to focus on component stewardships and technical debt
reduction to drive quality improvements
Re-aligned the organizational structure
and delivery teams
• Product and technology is reorganized into
~50 cross-functional scrum teams
• Each scrum team has ownership of a
platform service / application and is
responsible for a set of platform KPI’s
Enabled an end-to-end agile process
• Prioritization, Intake, Program, Product,
Software and Release Processes are all
agile
• Enable releases to become a non-event
and occur every two weeks
• Rapid experimentation of testable
product innovation and features
Adopted new agile management
and automation tools
• Agile ALM tooling for managing
programs and product backlogs
• Continuous integration for
automating continuous builds,
testing and deployment
1
2
3 4
Case Study: 1000+ product and engineering division within a 10,000+ FSI IT organization
Principles of Componentization:
Define bounded contexts to enable:
 Single-purpose assembly components
 Social architecture build around explicit
contracts
 Independently releasable components
Expose an open source model to provide:
 Developers to freely modify code and deploy
 Stewardship and quality to protect developers
from “breaking it”
 Encourage cross-training and knowledge
sharing
Establish a common product language in the
architecture so:
 The system becomes self-describing
 Business SMEs and Technologists can both
speak the same language
 Architecture always stays aligned with the
product
Componentization
Re-organizing the architecture based on a set of independent
cells that can bond together to deliver greater value
Responsive Web
Customer and Account
Management Platform
Partner Integration Services
Developers can
freely contribute to
other code bases
but must abide by
the cells “rules”
Stable teams are
organized within
the cell based on
single purpose
components
Cells can
independently
release
components in
their bounded
context
H
BeLi
MgNa
K Ca
Sc
The cells help
establish a common
language in business
terms
Y
Lu*
Componentization – Case Study
Customer Channels
Platforms Integrations
Platform Platform Platform Platform
Mobile Web IVR Open API Servicing
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
The How – Key Highlights
 Defined clear responsibilities and boundaries for each product that could be managed, evolved and supported by one or
more scrum teams
 “Componentized” the architecture and code, each product had a set of components which can be independently tested
and released
 Refactored each component to avoid direct database calls and instead go through platform services
 Applied a package manager system to help manage and automate packaging and versioning of each component
Component
Component
Component
Component
Component
Component
Component
Component
Services
Component
Component
Abstracted product architecture from the case study
 Develop a coherent view of your architecture and start thinking how it can be
componentized into products
 Refactor the design and code as new features come in to increase decoupling and
expose discrete APIs for greater reuse
 Leverage domain driven design to start expressing the design and code in a
manner that traces easily to business and product language
What you can do tomorrow
Where to learn more:
 Lean Architecture: for Agile Software Development – James O. Coplien
 Working Effectively with Legacy Code – Michael Feathers
 Domain Driven Design – Eric Evans
Building Software Product
Teams for the Architecture Establishing effective software product
management requires software product teams
that:
Sets the product vision for their components by:
 Owning and managing a product backlog
 Defining and driving product KPI’s to help
prioritize value and enable validated
experimentation
Owns, manages and deploys its code separately to:
 Deploy on its own schedule (no more “whole
stack deployments”)
 Build and execute its own test harnesses and
test cases
 Drives down its own technical debt and is
responsible for maintaining reuse, cohesion
and coupling with other components
Using cadence and flow to coordinate with other
software product teams to:
 Establish a big picture product architecture
view
 Pairs with horizontal specialist teams
 Plans and synchronizes with other teams
regularly
Establishing a network of software product teams that each
own product outcomes, technical quality and coordinate on a
cadence
Horizontal teams pair
with software product
teams to shape end-to-
end concerns
(architecture, UX,
features)
Each team is empowered to set their
own product vision and manage their
priorities via a product backlog
The product quality (customer features,
operational quality) are managed, tested
and deployed by the team
Based on flow, horizontal teams
and software product teams
collaborate to shape new work
Regular
cadences help
teams
coordinate
dependencies
Building software product ownership – Case Study
Feature/Experience
Designers
The How – Key Highlights
 Established product owners for each scrum team based on the product architecture and formed stable teams by
component
 Formed “one product engineering team” consisting of program owners, feature/experience designers, product owners,
and architects to work together to envision and deliver, resulted in ~50 scrum teams
 Empowered each team to work with other teams to decouple their code based on the product architecture and to drive
their own technical debt (60/40 backlog ratio)
Architecture
Program Owners
Product Owners and Scrum Teams
 Collect and funnel work into a single product backlog where there is a well defined
product boundary
 Establish a “technical product owner” as a proxy and increase collaboration with
business partners to increase communication and awareness of the role and value
 Work closer with stakeholders to express requirements in smaller units of value
(e.g. user stories) that can be developed and tested more independently earlier in
the process
What you can do tomorrow
Where to learn more:
 Agile Product Management with Scrum: Creating Products that Customers Love –
Roman Pichler
 Principles of Product Development Flow: Second Generation Lean Product
Development – Donald G. Reinersten
 User Stories Applied: For Agile Software Development – Mike Cohn
Taking Continuous
Integration to the Enterprise
Level
To succeed in delivering speed @ scale requires a
set of disciplined technical practices:
Establishing an enterprise wide “definition of
done” (DoD):
 Integration testing and other forms of testing
needs to be pushed upfront into the DoD
 Nothing goes to the mainline or release
without meeting DoD
Shift from managing dependencies to deferring
them:
 Traditional code management practices needs a
rethink, feature branches vs main line and
feature toggles/switches to minimize conflicts
across developers while balancing early
integration
 Stubbing/mocking becomes more important so
teams can develop more independently of each
other
Automation needs to be norm:
 Automate the end-to-end development, build,
testing, testing architecture and engineering
plays a critical role in enabling components and
products to be independently testable
 Test automation becomes a first class citizen
where test frameworks and code becomes just
as important as feature code
Enterprise level continuous integration provides the tracks for
the agile release train to run and get to continuous delivery
Every two weeks, the train picks up product increments that
are “code clean” and tagged for pushing to production
In each sprint, teams work to get stories “code clean” and tag it
for release, it’s up to each team to make sure quality passes in the
CI process
Continuous integration process runs for
every commit and kicks-off a series of
tests to verify “code clean” is met
Features that are have dependencies with other teams
that are not ready for release yet, are toggled “off” or
dialed up or down for experimentation
S1 S2
R1 R2
Taking Continuous Integration to the enterprise level – Case
Study
The How – Key Highlights
 Leveraging componentization and extending coding practices to always use interface based development allowed for
easier mocking/stubbing to support testing in isolation
 Mainline code management was used with feature toggles/switches and API versioning that enabled each component to
merge frequently and push to production without being impacted by dependencies
 An enterprise wide baseline “definition of done” (code clean) was defined and followed that forced full testing to be
completed by the end of each sprint/iteration before work could be accepted
 A dedicated build environment was setup based on a testing architecture that supported full build automation,
continuous integration, notifications and reports on meeting the “definition of done” based on coverage and quality
metrics
 Setup or leverage an existing continuous integration server and work with your
team to implement
 Work with your team to define a baseline “definition of done” that is used for the
continuous integration process to assess go/no-go
 Refactor code to be more interface-based and work with others to improve and
speed up the test automation coverage for the end-to-end range of testing
including unit, functional and integration testing
What you can do tomorrow
Where to learn more:
 Continuous Integration: Improving Software Quality and Reducing Risk – Paul M.
Duvall, Steve Matyas, Andrew Glover
 Continuous Delivery: Reliable Software Releases through Build, Test and Deployment
Automation – Jez Humble, David Farley
 How Google Tests Software – James A. Whittaker, Jason Arbon, Jeff Carollo
Bringing it all back together and how to get started
Agile @ Speed and Scale is being
done by organizations out there by
taking an agile architecture
approach. Disney can benefit from
advancing the state of agile maturity
today in the organization and taking
a step wise incremental approach to
agile adoption.
1) Adopt agility and speed as a culture by looking at how the three practices can be
implemented
 Componentization
 Establishing software product teams for the architecture
 Taking continuous integration to the enterprise level
2) Refine and increase the scale of these practices across the organization to achieve
operational stability based on fast and frequent releases
3) Leverage the high speed agile engine and adopt a rapid experimentation
approach to think and act more like a lean startup
Thank You!
Q&A
Alexis Hui
Twitter: @alhui
Email: alhui@deloitte.ca
Blog: www.agileconsulting.blogspot.com

Agile architecture made real

  • 1.
    Agile Architecture MadeReal Re-architecting the organization for speed - Building a fast and flexible architecture across 50 scrum teams
  • 2.
    Introductions and whywe are here Alexis Hui  Co-lead the Deloitte LEAN practice – Agile Advisory and Transformation  10+ years of agile experience - doing, advising and coaching  Helping organizations make agile work @ scale (100’s-1000’s) My goal is to share with you how agile can work at speed and scale based on a case study to make it real
  • 3.
    Who needs architecture? Scrum XP Kanban Agile Modeling Method 100’s1000’s # people Ok… something is missing My toolkit worked great to help organizations become more agile… until we started hitting larger organizations
  • 4.
    Architecture is atthe heart of making agile work @ scale Move Fast and Break Things Self-organizing teams EXPERIMENT. FAIL. LEARN. REPEAT.
  • 5.
    Speed and scale– Oxymoron? Large platform  3M+ accounts  $2B+ funds  5.5M+ transactions / mth  200+ product features with multiple products Distributed organization  1000+ people  60-70% outsourced  7+ locations, 5 major delivery centers  5 time zones Lots of Partners  Co-branding  FSI’s  Enterprise back-ends  Enterprise servicing New Speed Demands  Handle constantly changing programs and business development contracts  Heavily regulated industry with frequent changes in regulations  Highly competitive space with “fast” competitors  Enormous pressure to drive big revenue growth and acquire customers  Desire to deploy and release every two weeks New business unit spun up 4 years ago to drive new revenue streams for a much larger traditional parent FSI organization…
  • 6.
    Rethinking the architecture– not just software What is the real product?  Reframing the world, the product is the platform How do we scale our architecture to the next level?  Organize teams around the product architecture and let teams do the work for you How do we release faster than ever?  Re-architect everything we have to release every two weeks …organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations – M. Conway
  • 7.
    Agile Architecture inAction Refactored their platforms with componentization in mind • Platform refactored into a discrete set of products and platform services • Agile is enabling teams to focus on component stewardships and technical debt reduction to drive quality improvements Re-aligned the organizational structure and delivery teams • Product and technology is reorganized into ~50 cross-functional scrum teams • Each scrum team has ownership of a platform service / application and is responsible for a set of platform KPI’s Enabled an end-to-end agile process • Prioritization, Intake, Program, Product, Software and Release Processes are all agile • Enable releases to become a non-event and occur every two weeks • Rapid experimentation of testable product innovation and features Adopted new agile management and automation tools • Agile ALM tooling for managing programs and product backlogs • Continuous integration for automating continuous builds, testing and deployment 1 2 3 4 Case Study: 1000+ product and engineering division within a 10,000+ FSI IT organization
  • 8.
    Principles of Componentization: Definebounded contexts to enable:  Single-purpose assembly components  Social architecture build around explicit contracts  Independently releasable components Expose an open source model to provide:  Developers to freely modify code and deploy  Stewardship and quality to protect developers from “breaking it”  Encourage cross-training and knowledge sharing Establish a common product language in the architecture so:  The system becomes self-describing  Business SMEs and Technologists can both speak the same language  Architecture always stays aligned with the product Componentization
  • 9.
    Re-organizing the architecturebased on a set of independent cells that can bond together to deliver greater value Responsive Web Customer and Account Management Platform Partner Integration Services Developers can freely contribute to other code bases but must abide by the cells “rules” Stable teams are organized within the cell based on single purpose components Cells can independently release components in their bounded context H BeLi MgNa K Ca Sc The cells help establish a common language in business terms Y Lu*
  • 10.
    Componentization – CaseStudy Customer Channels Platforms Integrations Platform Platform Platform Platform Mobile Web IVR Open API Servicing Component Component Component Component Component Component Component Component Component Component The How – Key Highlights  Defined clear responsibilities and boundaries for each product that could be managed, evolved and supported by one or more scrum teams  “Componentized” the architecture and code, each product had a set of components which can be independently tested and released  Refactored each component to avoid direct database calls and instead go through platform services  Applied a package manager system to help manage and automate packaging and versioning of each component Component Component Component Component Component Component Component Component Services Component Component Abstracted product architecture from the case study
  • 11.
     Develop acoherent view of your architecture and start thinking how it can be componentized into products  Refactor the design and code as new features come in to increase decoupling and expose discrete APIs for greater reuse  Leverage domain driven design to start expressing the design and code in a manner that traces easily to business and product language What you can do tomorrow Where to learn more:  Lean Architecture: for Agile Software Development – James O. Coplien  Working Effectively with Legacy Code – Michael Feathers  Domain Driven Design – Eric Evans
  • 12.
    Building Software Product Teamsfor the Architecture Establishing effective software product management requires software product teams that: Sets the product vision for their components by:  Owning and managing a product backlog  Defining and driving product KPI’s to help prioritize value and enable validated experimentation Owns, manages and deploys its code separately to:  Deploy on its own schedule (no more “whole stack deployments”)  Build and execute its own test harnesses and test cases  Drives down its own technical debt and is responsible for maintaining reuse, cohesion and coupling with other components Using cadence and flow to coordinate with other software product teams to:  Establish a big picture product architecture view  Pairs with horizontal specialist teams  Plans and synchronizes with other teams regularly
  • 13.
    Establishing a networkof software product teams that each own product outcomes, technical quality and coordinate on a cadence Horizontal teams pair with software product teams to shape end-to- end concerns (architecture, UX, features) Each team is empowered to set their own product vision and manage their priorities via a product backlog The product quality (customer features, operational quality) are managed, tested and deployed by the team Based on flow, horizontal teams and software product teams collaborate to shape new work Regular cadences help teams coordinate dependencies
  • 14.
    Building software productownership – Case Study Feature/Experience Designers The How – Key Highlights  Established product owners for each scrum team based on the product architecture and formed stable teams by component  Formed “one product engineering team” consisting of program owners, feature/experience designers, product owners, and architects to work together to envision and deliver, resulted in ~50 scrum teams  Empowered each team to work with other teams to decouple their code based on the product architecture and to drive their own technical debt (60/40 backlog ratio) Architecture Program Owners Product Owners and Scrum Teams
  • 15.
     Collect andfunnel work into a single product backlog where there is a well defined product boundary  Establish a “technical product owner” as a proxy and increase collaboration with business partners to increase communication and awareness of the role and value  Work closer with stakeholders to express requirements in smaller units of value (e.g. user stories) that can be developed and tested more independently earlier in the process What you can do tomorrow Where to learn more:  Agile Product Management with Scrum: Creating Products that Customers Love – Roman Pichler  Principles of Product Development Flow: Second Generation Lean Product Development – Donald G. Reinersten  User Stories Applied: For Agile Software Development – Mike Cohn
  • 16.
    Taking Continuous Integration tothe Enterprise Level To succeed in delivering speed @ scale requires a set of disciplined technical practices: Establishing an enterprise wide “definition of done” (DoD):  Integration testing and other forms of testing needs to be pushed upfront into the DoD  Nothing goes to the mainline or release without meeting DoD Shift from managing dependencies to deferring them:  Traditional code management practices needs a rethink, feature branches vs main line and feature toggles/switches to minimize conflicts across developers while balancing early integration  Stubbing/mocking becomes more important so teams can develop more independently of each other Automation needs to be norm:  Automate the end-to-end development, build, testing, testing architecture and engineering plays a critical role in enabling components and products to be independently testable  Test automation becomes a first class citizen where test frameworks and code becomes just as important as feature code
  • 17.
    Enterprise level continuousintegration provides the tracks for the agile release train to run and get to continuous delivery Every two weeks, the train picks up product increments that are “code clean” and tagged for pushing to production In each sprint, teams work to get stories “code clean” and tag it for release, it’s up to each team to make sure quality passes in the CI process Continuous integration process runs for every commit and kicks-off a series of tests to verify “code clean” is met Features that are have dependencies with other teams that are not ready for release yet, are toggled “off” or dialed up or down for experimentation S1 S2 R1 R2
  • 18.
    Taking Continuous Integrationto the enterprise level – Case Study The How – Key Highlights  Leveraging componentization and extending coding practices to always use interface based development allowed for easier mocking/stubbing to support testing in isolation  Mainline code management was used with feature toggles/switches and API versioning that enabled each component to merge frequently and push to production without being impacted by dependencies  An enterprise wide baseline “definition of done” (code clean) was defined and followed that forced full testing to be completed by the end of each sprint/iteration before work could be accepted  A dedicated build environment was setup based on a testing architecture that supported full build automation, continuous integration, notifications and reports on meeting the “definition of done” based on coverage and quality metrics
  • 19.
     Setup orleverage an existing continuous integration server and work with your team to implement  Work with your team to define a baseline “definition of done” that is used for the continuous integration process to assess go/no-go  Refactor code to be more interface-based and work with others to improve and speed up the test automation coverage for the end-to-end range of testing including unit, functional and integration testing What you can do tomorrow Where to learn more:  Continuous Integration: Improving Software Quality and Reducing Risk – Paul M. Duvall, Steve Matyas, Andrew Glover  Continuous Delivery: Reliable Software Releases through Build, Test and Deployment Automation – Jez Humble, David Farley  How Google Tests Software – James A. Whittaker, Jason Arbon, Jeff Carollo
  • 20.
    Bringing it allback together and how to get started Agile @ Speed and Scale is being done by organizations out there by taking an agile architecture approach. Disney can benefit from advancing the state of agile maturity today in the organization and taking a step wise incremental approach to agile adoption. 1) Adopt agility and speed as a culture by looking at how the three practices can be implemented  Componentization  Establishing software product teams for the architecture  Taking continuous integration to the enterprise level 2) Refine and increase the scale of these practices across the organization to achieve operational stability based on fast and frequent releases 3) Leverage the high speed agile engine and adopt a rapid experimentation approach to think and act more like a lean startup
  • 21.
    Thank You! Q&A Alexis Hui Twitter:@alhui Email: alhui@deloitte.ca Blog: www.agileconsulting.blogspot.com