We are in a very interesting space where we own and maintain an estate of 30+ services (mix of java, clojure). Apart from the usual challenges with evolutionary architecture, the problem domain and operational structures of the organization constraint us to only evolve on a yearly cadence. This is the story of the things we managed to do well and not so well, things we planned for and things we missed planning for, the impact of tech hype cycles and the implications of ~300 different hands.

1. Evolution : Experience
Report
Shodhan Sheth

2. Techy at heart, currently tech principal
@thoughtworks
Who am I

3. Context of client
● UK Public sector
● Administers a set of financial products offered to UK residents
● 40-50 products depending on how you count
● Slave to policy, regulatory [& political] needs
● Have been personally helping them for a total of ~3 years in an attempt to
re-platform the core systems

4. Organizational Constraints
● Single software body to execute policy intent of 4 different policy entities
& 4 different operational entities
● Tied to public sector budgeting processes and cycles
○ Budgets approved ~1 year before start of FY
● Return on Investment model dependent on high proportions of reuse of
tech across all products

5. Baggage
● Large legacy estate [>15 years old tech, not supported by authoring orgs]
● Heavy change management process
● Shared, snowflake environments
● Emphasis on heavy manual end-to-end testing before releases
● On premise, manually configured infrastructure

6. What have we achieved so far

7. Technology oasis
● Own & maintain 30+ services using languages, libraries and approaches
relevant today
● Contributed some components [libraries helping implementing
organization wide CFRs – authentication, logging, tracing, etc] back to
other parts of the organization
● Automated pipelines that can help move code from commit to live in ~2h
● Standup a skeleton service [including configuration management] in ~1d
● Service 10% of live customer volume [rest 90% is on legacy system]

8. What's special ?

9. Long Tail

10. Long Tail
Consumption
Products

11. Long Tail – Movie rental business
Views
Movie titles

12. Long Tail
Views
Movie titles
Blockbuster

13. Long Tail
Views
Movie titles
Netflix

14. Long Tail
Numberofcustomers
Customer circumstances
This
place

15. What does that mean
● Need to cover all types of customer circumstances for the service being
offered
● Really stretches the ability to reuse building blocks
● Forces one to build for edge cases/ exceptions first as that becomes the
core USP

16. Impact
● Influences product evolution strategy
○ Need to cover all types of customer circumstances for the service being offered
● Forces one to build for edge cases/ exceptions first as that becomes the
core USP

17. Policy Policy Policy

18. Policy driven delivery
● Every year a new policy spec is published by the governing body, which
builds on top of an existing policy spec
● At any particular point of time there are 300-400 policy cohorts live
○ Every year some of the oldest policy sets become defunct, while new ones get added
for new cohorts
● A policy cohort can have a life of 3-16 years depending of the product
● Policy cohorts are guarded by “transitional protection” clauses

19. Awesome

20. Good

21. Maybe Not Good
● It isn’t really like
this, but
something like
it**
● Its difficult to get
this picture for
this client as we
were multi repo

22. Why ?
● Business constraints on changing the outcome of a policy cohort
○ Organizational fear of technology lead change/ refactor leading to change in already
delivered policy outcome
○ Organizational fear of automation suites to be enough of a safety net
● We dint want to constraint ourselves to sticking to the first
implementation model we had created [we had learnt so much since]
● Luckily the first policy cohorts we implemented had a life of ~4 years
○ So they would die a natural death soon

23. Enduring solution
● Segregating policy spec implementation and surrounding operational
requirements
● Strategy patterns that allow for different implementations for known
variances in policy cohorts [~50 dimensions from historical evidence and a
domain experts view]
● Influencing organizational behavior by showing concrete evidence of
change supported by automated safety nets

24. The human problem

25. In the last 5 years
● Many [~10] CXOs changed
● Many [~20] Tech Leads/
Architects changed
● Many [~200] developers
changed
● The tech landscape changed

26. In the last 5 years
● Many [~10] CXOs changed
● Many [~20] Tech Leads/
Architects changed
● Many [~200] developers
changed
● The tech landscape changed
● Leadership vision changed
● Architectural philosophies
changed
● Low level code preferences
changed
● What was best practice when
we started is frowned upon
now

27. Humans are unique snowflakes
● Consensus amongst a given set of people is hard
○ Have to rely on authority models a lot of times to drive decisions
○ Creates some tension with providing teams autonomy
● Adding people churn to that model makes it harder
● It becomes exponentially harder when the churn is at the top because of
the heavy reliance on authority models
● Caused some unintended evolution paths

28. What we could have done better
● Don’t really have a good answer
● The notion of fitness functions* for each service to guide evolution of key
technical and business tenants sounds promising
○ Though automating some of those fitness functions seemed hard on first thoughts
○ But maybe even documenting those tenants could have forced more deliberate
thought

29. THANK YOU
For questions or suggestions:
Shodhan Sheth
shodhan.sheth@thoughtworks.com