Architecture Entropy is a term used to describe the slow design erosion away from the structured, governed and organised towards a more disordered state.
Regardless of how well designed a computer system is, it will be subjected to the laws of Architecture Entropy.
This presentation attempts to define the term and start some thinking on how to tackle it.
1. Simon Greig, Executive IT Architect, IBM Global Business Services
November 2015
Architecture Entropy A
2. About the Author
Simon is an experienced IBM Executive IT Architect with 20 years experience in
designing and delivering complex projects
He has been working on complex systems integration projects since 1999 and
over the years have been immersed in SOA, ESB and more recently cloud,
mobile and agile technologies
Over his career he has delivered projects worth cumulatively about US$2Bn
Simon came up with the title term a few years ago and has finally decided to do
something about it!
It is one person’s point of view on the subject…!
2
Simon Greig
Executive IT Architect
IBM Global Business Services
Europe
3. Contents
What is Architecture Entropy
Example Architecture Entropy in Action
Architecture Entropy Consequences
What can we do about it?
A
5. ar·chi·tec·ture (är′kĭ-tĕk′chər)
noun.
1. The art and science of designing and erecting buildings.
2. Buildings and other large structures
3. A style and method of design and construction
4. Orderly arrangement of parts; structure
5. The overall design or structure of a computer system or
microprocessor, including the hardware or software required to run it.
6. Any of various disciplines concerned with the design or organization of
complex systems
Source: http://www.thefreedictionary.com/architecture
6. noun.
1. Symbol S For a closed thermodynamic system, a quantitative measure
of the amount of thermal energy not available to do work.
2. A measure of the disorder or randomness in a closed system.
3. A measure of the loss of information in a transmitted message.
4. The tendency for all matter and energy in the universe to evolve toward
a state of inert uniformity.
5. Inevitable and steady deterioration of a system or society.
en·tro·py (ĕn′trə-pē)
Source: http://www.thefreedictionary.com/entropy
(in other words, the universe tends towards disorder rather than order…)
7. compound noun.
1. A measure of the disorder in a computing
system.
2. The inevitable and steady deterioration of a
computing system toward a state of disorder.
architecture entropy
8. A
Architecture Entropy
Architecture Entropy is a term used to describe the slow design erosion away from the
structured, governed and organised towards a more disordered state
Regardless of how well designed a computer system is, it will be subjected to the laws of
Architecture Entropy
Typically a well designed system will initially have a low entropy due to the structure and
architecture of the solution
Over time the system will be subjected to ‘entropy gain’ as the architectural and structural
integrity of the system are eroded
All systems in a single organisation will eventually reach equilibrium at a similar level of
entropy
Each organisation’s natural state of entropy will differ from organisation to organisation but it
will always reflect the principles and attitudes of the overall organisation management
Architecture Entropy gain cannot be avoided but the levels of entropy gain can be
minimised with appropriate governance and budgeting
10. A High Level Example
This example is based on real experiences, it is not based on one single enterprise but the
concepts and outcomes are real
11. As-Is Enterprise
Components
A snapshot of part of a
complex enterprise estate
Many connections between
many components leads to
complexity and change cost
12. To-Be Vision
An enterprise service bus
component has been added to
provide simplification of
connectivity
Components D and G & H
have been decommissioned
Structured, organised, tidy,
clean…
…Expensive
13. Eventual Entropy
State
During delivery it becomes hard to
justify altering legacy systems that
have been running for years without
issue
Some connections are rationalised
but others remain for operational
reasons
There are short term pressures to
deliver some benefit early
The ‘transition architecture’ is
complex but a later release will ‘tidy
things up’
Connections bypassing the ESB
are re-established because they are
quicker and cheaper in the short
term
14. A
Outcomes – any of this sound familiar?
1. The plan was to give the business what was needed as soon as possible and then tidy up
the IT in the next release. The cost of later releases couldn’t be justified and so didn’t
happen.
2. The additional IT complexity increased downstream costs and therefore “quicker” and
“cheaper” alternatives to following the strategy were championed by the funding
stakeholders.
3. The plan was based on rationalising and decommissioning legacy systems. However it
was discovered late on that there were many more dependencies on the legacy systems
and so it was determined to be too costly to decommission all of the legacy systems.
4. The short term “tactical solution” that was only intended to be live for a few months is now
many years old and requires a lot of effort to keep it running.
The result: The enterprise estate remained complex and expensive
16. Entropy gain is directly linked to an increase in costs
The higher the entropy gain, the higher the overall architecture entropy and the higher the
architecture’s relative operational costs
Typically entropy gain is caused by:
Tactical bolt ons rather than engineered extensions
Bypassing components in order to save time and then adding
additional function to compensate
Stovepiping
Short Circuiting
Duplication
Avoiding flexibility in order to solve the problem of the day rather
than the bigger picture
Hardcoding
Creating function in more than one place
17. Costs need to be balanced…
Low Cost to Operate
Low Cost to Change Low Cost to Build
Enterprise dilemma: “Which two do you want as you can’t have all three?”
18. Consequences to Balance
• Impact on operate costs: Risk of overall system fragility if “low cost”
means “corners were cut” or elements of the system were left to be
performed manually
• Impact on change costs: Possibility of functional duplication as it was
cheaper to ‘copy and paste’ function than it was to share and reuse
existing. Therefore increases the cost to change
• Impact on operate costs: An increase of overall system complexity to
accommodate the flexibility features
• Impact on build costs: Extra effort to design, build and (in particular) test
the flexibility features
• Impact on change costs: Potential inflexibility due to the run costs being
optimised around the ‘go live’ state of the system
• Impact on build costs: Increased levels of automation that requires
additional design, build and test effort
Low
Cost to
Build
Low
Cost to
Change
Low
Cost to
Operate
19. Costs
Entropy Gain over Time
Reaching Architectural Equilibrium
The Goal: Reaching a point where the architectural integrity of a system or enterprise is in
balance with the costs
Achievement of this goal is incredibly hard and arguably one of the holy grails of IT
We should still try our best to balance as best as we can
Architecture
Integrity Costs
Equilibrium point
21. The Level of “Entropy Gain” is Variable
Many factors determine the level of “entropy gain” of a system
– Strength of technical governance
– Size of the general investment budget
– Business’s attitude to the complexities of enterprise IT
– Organisational preference to ‘tactical’ vs ‘strategic’
– The ‘background level’ of complexity already inherent in the IT estate
An amount of gain is inevitable due to pressures on time and budget
A small amount of gain may be beneficial to allow a system to reach equilibrium
The amount of gain and downstream impact can be minimised with appropriate governance
and management
Ultimately it is the IT department’s relationship with the business stakeholders that
determines the entropy levels
22.
23. Measure
Measure
– The simplest way to measure entropy gain is to focus on
the downstream costs of a particular cost
– Don’t just focus the business case on the cost to
implement; look also at a portfolio of common business
change scenarios and the 5 year cost
– Research the actual long term ‘lights on cost’ that the
enterprise has accrued over time
24. Average Annual Cost
When comparing solution options and when ‘tactical’ vs ‘strategic’ consider the average
annual cost rather than the upfront cost when comparing options
Where
– A is the Average Annual Cost
– C is the estimated number of system changes in a year (constant for all options)
– T is the length of time the solution will be around for (constant for all options)
A = Build Cost + C(Average Change Cost) + T(Annual Cost)
T
25. Manage
Manage
– Strengthen governance of system change to
minimise the risk of short term changes causing
long term costs
– Create a change checklist to ensure that solution
designers are considering the full life cycle
changes
– Keep focus on the cost case for the solution
– Tightly manage deviations and exceptions from the
solution architecture as if the system was being
created from new
26. Minimise
Minimise
– Make sure that each solution release provides value to
the business and is not ‘just’ IT benefit
– Use establish facts based on history and current costs
– Use ‘tactical solutions’ with caution
– Have a strong exit plan to get off the tactical solution
– Calculate the full lifecycle costs of the tactical solution
– Overall though, be pragmatic!
– Every solution has an equilibrium point where the
balance between the architecture purity and the overall
costs is met
28. Architecture Entropy States
Strong business and technical
governance
Full lifecycle design considerations
Managed exception processes so that
exceptions to the standards can be
achieved with managed consequences
Decisions based on the TCO
Low Entropy
Medium to long term operational cost
increases
Incrementally slower and more
expensive to change systems
Risk of fragility in the enterprise
Progressively increasing operational
costs
High Entropy
!
GOOD! BAD!
29. Be Aware!
Architecture Entropy will always exist
Nothing can be done to prevent entropy gain
Awareness of the existence of Architecture
Entropy should help to minimise entropy gain
Invest effort to measure the impacts of
decisions, especially in the longer term
Use the measurements to manage better
outcomes
Minimise short term behaviours that can
negatively impact an enterprise’s Architecture
Entropy
A
30. Further research needed…
Writing this document concluded that there are more
questions to anwer:
– Is it possible to consistently measure the relative
entropy state of an architecture?
– Is it possible to measure the “architecture half-
life” to predict a either a point in time:
Where the costs of the architecture outweigh
the benefits?
When the integrity of an architecture is
compromised beyond a point where it is
wasteful to apply full governance to it?