How do we integrate agile delivery with the complexities of the legacy enterprise environment? Agile is fast moving and takes no prisoners, yet in an enterprise system delivery context the agile delivery could be be dependent on the legacy un-agile enterprise that holds the data and business processing logic. How are these diverse elements integrated? This is one person's point of view...

1. Balancing Agile Delivery with the Enterprise Estate
Simon Greig, Executive IT Architect, IBM Global Business Services
April 2015
Agile Enterprise Integration

2. About the Author
 Simon is an experienced IBM Executive IT Architect with 20 years experience in
designing and delivery complex projects
 He has been working on complex systems integration projects since 1999 and
over the years have been immersed in SOA, ESB and more recently cloud,
mobile and agile technologies
 Over his career he has delivered projects worth cumulatively about US$2Bn
 His current role in IBM is as a technical leader in the Public Sector business
within IBM Global Business Services Europe
 This presentation was created following many conversations with clients and
colleagues about how to integrate agile and legacy
 It is one person’s point of view on the subject…!
2
Simon Greig
Executive IT Architect
IBM Global Business Services
Europe

3. Enterprise Integration Context
3

4. Enterprise System Context
User Experience Transaction Management
Legacy Integration Data Integration
 Agile is an excellent fit to the “user tangibles” of creating the stuff that
users can see and use such as the user experience
 There are additional challenges with complex systems that need to be
managed
– Complex interdependencies between components
– External dependencies and constraints (e.g. legacy systems, 3rd party
interfaces)
– Agreement of interfaces / Data format discussions and agreements
– Availability of test environments
– Mismatches between components
– Cost of integration and non-functional testing
 And common problems with running enterprise systems that need to be
addressed
– Data quality issues
– Software bugs
– Complex middleware configuration issues
– External interface data and availability problems
– Ensuring data integrity at all times
– How to do point in time backups without impacting live service
– How to minimise patching impacts and subsequent system downtime
 Assuming that new agile systems will simply avoid or eliminate
complexity is dangerous because external factors will drive up complexity
and introduce significant constraints
 User Tangibles are just the tip of the iceberg…
4
Agile works best with
this bit
Enterprise
Solution

5. 5
USER EXPERIENCE IS THE
TIP OF THE ICEBERG

6. 6

7. Integration Spectrum
7

8. An Integration Spectrum
8
Agile = Opportunity
• Contemporary interfaces
• Relatively small number of interfaces
and components to integrate
• Single or few party delivery
• Consistency
• Few external dependencies
Enterprise = Complexity
• Mixture of technologies
• Large number of interfaces and
components to integrate
• Many parties involved, some of
whom are unable to support many
changes due to their own release
cycles
• Complex service management
environment
• Many stakeholders
• Many constraints
• Complex external dependencies
System Integration solutions follow an integration spectrum from simple to complex.
Each type of integration is different and the further something moves away from
standalone disconnected systems the more complex and difficult it becomes

9. However there is room for both:
9
AgileServicesEnterpriseSystems
Agile services:
• Able to be iterated quickly
• Fast delivery of changes
• Doesn’t have to be right first time
• Minimal or no inter sprint dependencies
• Little or no external dependencies
Enterprise Systems:
• Long lead time for changes
• Integration testing needs to be planned in
for a long time
• Potentially lots of upstream and
downstream implications to manage
• Expensive if not right first time
• Complex external dependencies
• Custodians of high value business
assets:
• Enterprise Master Data
• Complex Business Transactions
• External Interfaces
• History

10. However there is room for both:
10
AgileServicesEnterpriseSystems
Agile services:
• Able to be iterated quickly
• Fast delivery of changes
• Doesn’t have to be right first time
• Minimal or no inter sprint dependencies
• Little or no external dependencies
Enterprise Systems:
• Long lead time for changes
• Integration testing needs to be planned in
for a long time
• Potentially lots of upstream and
downstream implications to manage
• Expensive if not right first time
• Complex external dependencies
• Custodians of high value business
assets:
• Enterprise Master Data
• Complex Business Transactions
• External Interfaces
• History
What do we do if
agile services
need access to
legacy enterprise
transactions and
data?

11. REBUILD AS AGILE?
 Many legacy systems are core to the operation of
the business
 The detail of how the legacy systems is not always
understood leading to the risk that a rebuild
doesn’t fully meet the business need
 An incremental rebuild may not be practical if the
business still needs to operate
UNLOCK SOMEHOW?
 Is it possible to trigger transactions and/or access
data via an existing interface or API?
 If not, how soon can a new interface be created?
The lead time is likely to be high.
 How will the integration be managed?
 How will we have confidence that the change to
the legacy system will be the right one as we might
not have time to make the change twice?
11
What do we do if agile
services need access
to legacy transactions
and data?
1 2

12. REBUILD AS AGILE?
 Many legacy systems are core to the operation of
the business
 The detail of how the legacy systems is not always
understood leading to the risk that a rebuild
doesn’t fully meet the business need
 An incremental rebuild may not be practical if the
business still needs to operate
UNLOCK SOMEHOW?
 Is it possible to trigger transactions and/or access
data via an existing interface or API?
 If not, how soon can a new interface be created?
The lead time is likely to be high.
 How will the integration be managed?
 How will we have confidence that the change to
the legacy system will be the right one as we might
not have time to make the change twice?
12
This requires a system
integration programme solution
that supports both legacy and
agile development approaches
What do we do if agile
services need access
to legacy transactions
and data?
1 2

13. Platform & Services
Agile Services
Enterprise Platform
• Innovation based
• Some strategic
• Some short lived pilots and PoCs
• Limited dependencies
• Potentially reduced SLAs
• Built upon the ‘platform’ of APIs offered by the
enterprise
• Can be ‘hardened’ to Enterprise Platform Services
• Service Managed (ITIL)
• SLAs for performance and availability
• Release Managed
• Solid
• Robust
• Dependable
• Hardened
• Accessed via APIs

14. Systems Integration Framework
14

15. Complex Systems Integration Framework
 In order to do complex systems integration combining new agile solutions with legacy and 3rd
party systems then a systems integration release framework is required
 The pace of the release is determined by the slowest participant as everything [usually] has to
be delivered into production together.
 Each system integration release needs to follow the structure defined below:
Enterprise Architecture
Release Governance
Application Delivery
Integration Delivery
Infrastructure Delivery
Work out
what it
needs to
do and
how it will
be done
Bring it all
together
Make sure
it works
Operate

16. 16
Application Delivery
Integration Delivery
Infrastructure Delivery
Work
out what
it needs
to do
and how
it will be
done
Bring it
all
together
Make
sure it
works
Operate
Phase Name INCEPTION
Description The upfront planning and solution outline phase used to work out the high level scope of the
solution, the benefits, outline cost and delivery timeframe.
Output • Delivery plan
• E2E solution design
• Initial requirements backlog
Example Activities • Determine the business goal and user needs mapped to the business strategy
• Agreeing outline objectives of the delivery and high level requirements
• user stories, epics, business outcomes, traditional requirement statements,
NFRs, etc
• Creating the E2E outline solution
• Technology selection
• Objective consideration of open source and commercial software options
• Buy vs build decisions for components
• Identification of major internal and external dependencies
• Identification of major risks and initial mitigation options
• Identification of external constraints
• Selection of delivery partners
• Specialist vendors
• SMEs
• Selection of delivery method and associated tools
Enablers • Specialist system integration methods and tools
• SMEs
• Open source software

17. 17
Application Delivery
Integration Delivery
Infrastructure Delivery
Work
out what
it needs
to do
and how
it will be
done
Bring it
all
together
Make
sure it
works
Operate
Phase Name DELIVERY
Description Multi-stream delivery of the release providing the application, integration and infrastructure
solutions using a variety of delivery methods and styles best suited to each situation.
Output • Design documentation
• Unit tested code (some of which will have already been integrated)
• Unit tested infrastructure
• Test scripts (automated and manual)
• Test data
• Test stubs
Example Activities • Agile delivery of components (in particular those with user facing interfaces)
• Negotiations and information sharing with with 3rd party interface suppliers
• Integrated data model development and rollout
• Application infrastructure services development
• Infrastructure build and testing of resilience, deployment scripts, security, backups, etc
• Business logic and rules discovery workshops (if not concluded in discovery)
• Configuration management and release management processes firmed up
• Construction/procurement/provisioning of dev, test and production environments
Enablers • Agile delivery
• DevOps automation
• Cloud based IaaS/PaaS provisioning
• Design patterns

18. 18
Application Delivery
Integration Delivery
Infrastructure Delivery
Work
out what
it needs
to do
and how
it will be
done
Bring it
all
together
Make
sure it
works
Operate
Phase Name INTEGRATION
Description The coming together of all components from all development streams and 3rd parties in order
to create the full system for the first time. Followed by a series of shakedown tests to ensure
that all of the joints are sound and that the system wide non-functional aspects (system
resilience, security, performance) meet expectations.
This stage is essential in systems where there is a legacy or non-agile 3rd party aspect that
cannot keep up with agile delivery and continuous integration.
Output • A ready to go live system that meets the business and user needs.
Example Activities • The multi-party development streams working together to integrate with each other
• Dependencies and constraints may have limited some components to have
been delivered sooner
• E2E integration testing through repeatable test scenarios
• Both user driven and external system interface driven
• System wide resilience testing
• Typically done once at the end of a release as it is expensive and intrusive
(e.g. requires dedicated access to an environment and physical visits to the
datacentre)
• E2E performance testing
• Complex systems typically exhibit unusual or unpredicted behaviour when put
under load
• User Acceptance testing
Enablers • Performance engineering
• Test data creation automation

19. 19
Application Delivery
Integration Delivery
Infrastructure Delivery
Work
out what
it needs
to do
and how
it will be
done
Bring it
all
together
Make
sure it
works
Operate
Phase Name OPERATE
Description The live running of the system by the business and on-going maintenance and enhancement
of the system.
Ongoing • Keeping the system meeting service levels
Example Activities • System monitoring
• Problem analysis
• Incident management & resolution
• Minor enhancements
• Capacity monitoring and forecasting
• Performance monitoring and forecasting
• Technology refresh
• Patching
• Security monitoring and alert management
Enablers • Service management processes
• DevOps style management where the development team do 1st, 2nd and 3rd line support
of the system rather than a service management team

20. Bringing it all together: Complex Systems Integration Delivery Framework
20
Inception Delivery Integration
Operation
Define Architecture Roadmap
Procurement Approach
Business Strategy
High Level Requirements
E2E Delivery Planning (Dependency mgt, risk mgt, issue mgt, integrated planning, change mgt, etc)
E2E Delivery Tech Governance (Design assurance, troubleshooting, making sure it will work at the end, etc)
Integrated Tooling Management & Maintenance (Reqs, Design models, defects, source code control, document repository, config mgt, change mgt, incident mgt, etc)
Environment Management, scheduling and deployments
Business Change Planning
Service Management Prep
E2E High
Level Design
Sprint
Zero
S/W
Selection
H/W &
Hosting
Selection
Ready for full
system
integration
gate
Supplier 1
Pla
n Buil
d Tes
t
Fix
Deploy
Waterfall Development
Legacy Change
Agile Sprints
Supplier 2
Supplier 3
Operational
AcceptanceTest
Multi-PartyComponentIntegrationTest
Multi-PartyE2EFunctionalSystem
Test
Non-Functional System Test
(Performance, Security and Resilience)
E2E
System
Integration
Test
User
Acceptance
Test
System Monitoring
E2E Service Management
Incident Management
AMS
Infrastructure Management
Capacity Planning
On-going Projects
(Continuous Improvement & Business Change)
Inception Delivery Integration Operation
Enterprise Architecture and Governance
e.g. infrastructure
Solution
Outline

21. Key Enablers
21

22. Agile Delivery
22
Point of View Agile needs to be partnered with engineering in order to create solutions with solid non-functional basis.
Systems engineering is needed in order to meet service levels and also integrate with the legacy (non-
agile) estate.
Agile may not be suitable for all elements of system delivery. Such as:
• Integration interfaces with 3rd party systems
• Data processing based function
• Deliveries with a number of dependencies
Pros • Ability to demonstrate progress of a system to the stakeholders is very valuable
• More focus on the output of working code than process and documentation
Cons • Dependencies need to be resolved before each sprint – a large number of dependencies may lead to
blocked sprints
• Risks of rework and churn need to be mitigated to avoid major re-writes
• Requires input and time investment from the business
• Building incrementally without a solid infrastructure platform puts the overall system stability at risk
• The commercial and risk ownership implications of agile delivery in a complex multi-supplier
environment are yet to be understood

23. Open Source Software
23
Point of View Open source software is an attractive proposition due to the low acquisition cost. However software selection
needs to objectively select the right solution based on a number of factors:
• Acquisition cost
• Maintenance cost / Enterprise support costs
• Maintainability
• Longevity of the platform
• Marketplace skills
• Robustness
• Performance / Relative hardware needs
• Features/Innovation
Pros • Low start up costs
• Typically faster start up costs
• Many products are very mature, stable and enterprise class
Cons • A lot of hype and myths surround open source software. e.g. if you don’t like the function, you can change it.
Changes cost time and money and usually void enterprise support agreements
• Variance between packages on security, stability and performance (although this is improving quickly)
• Some minor legal risks around Intellectual Property
• Variable skills and support depending on the product – which can change over time as fashions change

24. Cloud
24
Point of View Going forward, solutions should be designed to work on commodity platforms such as Linux on x86.
Cloud offers an opportunity to reduce the time to delivery of infrastructure to projects but it is not necessarily a cost
saver. Cost saving may be possible if systems are hosted on a commodity public cloud such as IBM SoftLayer,
however data security will limit the extent to which public cloud offerings can be utilised. Many cloud suppliers,
including SoftLayer, offer private clouds which are more attractive to an enterprise client.
Commercial private clouds have an associated hardware and deployment cost that needs to be picked up by the
customer. Additionally there may be some short term scalability constraints if more hardware needs to be
deployed quickly.
Pros • Rapid provisioning of new nodes if the capacity exists
• Inherent flexibility offered by hardware virtualisation technology
• Some suppliers may offset the upfront charge with a higher service charge which may be helpful for budgeting
and a business case
Cons • There remains concerns with enterprise data security – more a perception than a reality
• Cloud may not necessarily provide a cost saving as enterprises [currently] mostly prefer the majority of systems
hosted on a private cloud with dedicated non-shared hardware and storage
• Modifications such as the addition of security components and specific network separation zones within an
environment may limit some of the cloud benefits such as dynamic provisioning

25. Conclusion
25

26. Conclusion
Agile can offer significant benefits to the
business, in particular in areas where the user
requirements are not fully understood
External dependencies can heavily constrain
agile delivery
Solution delivery will always go at the pace of the
slowest participant regardless of how fast some
participants can be
‘Full stack’ enterprise system development and
integration needs a hybrid approach between
agile and traditional in order to support a
combination of agile and traditional delivery
methods
26