Software Principles and Project Deadlines Don't have to be Polar Opposites.pdf

Software Principles and Project
Deadlines Don’t Have to be Polar
Opposites
By Craig Saunders

Classified as General
Legal
© Kindred Group plc 2021 2

Premise
“
As Software Engineer’s we pride ourselves to build high quality
software using the best industry practises and principles.
But what happens when you’re asked to deliver a project with
impossible timescales where a quick hacky solution is all that time
allows.
This presentation talks through such a scenario where and
how we managed to achieve the right solution but also met
the business deadline.
In addition it talks briefly about the key principles we
followed to achieve this feet.

Bio - Life
Software Engineer with 26 years experience
Started writing C with Vi, moving onto C++ and Java
Worked with Microservices for about 6 years
Worked predominately with distributed systems
Huge tech advocate – love playing with the toys

Bio – At Kindred
Job title is a Line Lead, but that’s not all what I do
Work in the strategic side of software development
Work closely with Architects, Stakeholders and POs
Help steer and mentor the development team(s)
I’ve sometimes been called the octopus – many hats

Bio Cont…
© Kindred Group plc 2021
Ultimately, I’m a geek who likes playing with the toys!!

Who is Kindred?
A tech company that operates in the online gambling space

Kindred Stats and Facts
01
Broke up the monolith in 2010
Kindred were an early adopter to Microservices and are now in position of refactoring their
microservice architecture. We also make use a lot of event-driven architecture.
02
310, and counting, microservices in production
We currently have 310 microservices which has tripled in the last 2 to 3 years. With the drive to
challenge with the best-in-class platform, we our ability scale with smaller purposeful services.
03
Strive for Continuous Delivery – 11k releases per year
Analysis has shown that the more releases we do the less system outages we see. With this we
strive, where regulation permits, frequent releasing on smaller pieces of development. We have a
zero-downtime policy too – only in exceptional cases which must be justified and approved.
04
Operate in a strict compliance and regulated industry
Due to the nature of the industry, we operate in, we are heavily regulated.
05
Deploy software onto Kubernetes
We are cloud agnostic and see Kubernetes as a key strategy to be able to deploy into new
markets at speed and be able to scale to needs of that market.

Kindred Stats and Facts
06
Team responsibility for monitoring and observability
Each team is responsible for the health, monitoring of its components – you
must look after your own babies. There are guidelines, but the team decides how
they want to be alerted and handle production incidents.
07
Team ownership for development and Agile process
Each team is responsible for how they deliver, provided it can align at the macro-
level. Any Agile process can be used but is typically SCRUM or Kanban.
08
Upwards of 1.8 million active customers
We have 1.8 active customer – determined over a quarterly period. During the
last World Cup, we had 1 million plus customer in one month.
09
Average of 45M+ wallet transactions per day
We see peaks of 60+ million wallet transactions per day, which is now double
what we achieved as an all-time-high during the World Cup of 2018. The
average is 45M+.
10
Send-out capability to target 3M+customer per day
Depending on what events are happening, our tech capability allows Marketing
to target upwards of 3M+ customers a day (especially around Advent).

Set the Scene

Sound Familiar?
“ I know this is a crazy
deadline but what can I do.
I’m told failure is not an
option.
- Team Lead
“ We have to hit this deadline.
Anything else is a nice to
have.
- Product Owner
“ We’ve a lot of money riding on
this product launch, we can’t
be late.
- Business Stakeholder
“ We’ve already paid for the
Marketing advertisement.
- Marketing Manager

Time to Cry?
The amount of
tech debt this
will incur will hurt
us in the future
We haven’t even
been able to remove
the debt since the
last hack job!!
How’s this
going to be
possible?
Will we ever have
time to develop
software properly

How Do We Get Here?
At times it’s like we are diametrically opposed

Blame Game? No
It’s not a blame game, we’re both in it to succeed

Let’s Get Down to the Tech

Domain Driven Design
01
Ubiquitous Language
A practise of building up a common, rigorous language between developers, users and
stakeholders to ensure we are all talking on a way that we all understand.
02
Bounded Contexts
A defined part of software where particular terms, definitions and rules apply n a consistent way.
03
Maintaining Model Integrity
Defining how different bounded contexts interact with each. E.g. conformist, shard kernel, open
host, anti-corruption layer, etc..
04
Isolating the Domain – Layer Architecture
Ensuring the business domain layer is separated supporting layers to surface the business
concepts, and business rules.
05
Continuous Integration
Continuous/constant communication between the team members –most notable t by
hammering out the ubiquitous language, In addition continuous testing of the bounded contexts
and how the interact with other bound contexts. .

Clean Architecture
01
SOLID Principles
Six key principles to help guide development in the right way. E.g., Single Responsibility, Open-
closed Principle, Liskov’s Rule of Substitution, etc.
02
Component Principles – and extension of SOLID
Principles which are similar to SOLID but at the component boundary. Such examples are
common closure principle, common reuse principle, reuse/release equivalence principle, etc
03
Setting Boundaries
Boundaries are lines that separate software elements. They separate things that matter from
things that don’t, i.e., high-level components from low-level components.
04
Separating the Layers – Port & Adapters
Layers are concentric and represent how fundamental (or high-level) components are. At the
core, we have the high-level policies, i.e., stable and abstract components encapsulating our
business rules. On the outer ring, we have the details, for example, unstable and concrete GUI’s..
05
Clean Code
Clean code helps toe surface the elements of the system so that everyone can understand the
wood from the trees.

So What Do We Do About It?
Let’s Dive In

Current System
Comms
Gateway
ADAPTER
Campaign
System
ADAPTER
Unica Adapter
ADAPTER
Audience
Platform
Send-out
Platform
Comms
Platform
CustomerSelectionEvent
The customer selection event is
fired every time a customer is
being targeted as part of the
campaign send-out.
CommunicationsGatewayMarketingPublication
The event that is fired to instruct the
communications platform to send comms to a
customer.

Current System – Focus Area
Campaign
System
ADAPTER
Send-out
Platform

This Wasn’t Going to Work
Campaign System
ADAPTER
The component had become a micro-monolith
Needed to support real-time send-outs, which it didn’t
We had numerous performance issues
Development took longer due to huge tech debt
Wasn’t compatible with our Kubernetes strategy

Target State
Campaign
Send-out
Snapshot
ADAPTER
Campaign
Association
ADAPTER
Test Accounts
Adapter
ADAPTER
Campaign Send-out Executor
ADAPTER
Campaign
Customer
Details
ADAPTER
Marketing
Compliance
ADAPTER
Send-out
Platform
Campaign
Model
ADAPTER
Campaign
Orchestrator
ADAPTER
Campaign
Scheduler
ADAPTER

Component Responsibilities
Marshalls the send-
out. It delegates
send-out actions to
the relevant
components and
decides the next
step in the flow
Used a locker for
the customer
details. It stores
personalisation
data and contact
details. It also
knows how to
obtain data.
Responsible for
determining if a
customer can be
targeted for
compliance reasons.
It relates to
communications
and bonuses.
Responsible for
storing a snapshot
of the key
information needed
for a campaign
send-out.
Campaign
Send-out
Executor
Campaign
Customer
Details
Marketing
Compliance
Campaign
Send-out
Snapshot

Component Responsibilities
It orchestrates the
campaign send-out.
It notifies all the
interested
components and
instructs them to
publish their
snapshots.
Records alarms that
are determined by
the Orchestrator.
Based on the
alarms set, it
notifies the
Orchestrator initiate
the send-out.
Responsible for
publishing the
campaign setup
snapshot to be
used during the
send-out.
Responsible storing
all the customers
that are targeted
during the
campaign send-out.
Campaign
Orchestrator
Campaign
Scheduler
Campaign
Model
Campaign
Association

Target State – Affected Areas
Campaign
Send-out
Snapshot
ADAPTER
Campaign
Association
ADAPTER
Test Accounts
Adapter
ADAPTER
ADAPTER
Campaign
Customer
Details
ADAPTER
Marketing
Compliance
ADAPTER
Send-out
Platform
Campaign
Model
ADAPTER
Campaign
Orchestrator
ADAPTER
Campaign
Scheduler
ADAPTER

Merged State
Send-out
Platform
Marketing Compliance
Campaign Customer
Details
Campaign Send-out
Snapshot
Campaign Send-out Executor Component

Shock Horror

Why Was This Faster to Develop?
28
Not enough time to ensure we could hit the performance
When you have component to component calls, you always introduce latency which can be
remedied, but we didn’t know if we had the time to do it and hit the deadline.
Development was faster due working with one component
As the four components worked closely together, it meant it was easier to setup locally and test
the end-to-end flow.
Limited Failure Scenarios
Component to component calls introduces reasons to fail, requiring a retry; whereas JAVA
service method call don’t. Although these can be dealt with, we didn’t have the time to address
them at the point
Our Release Process For New Components is Longer
Due to the due diligence checks that are required for new components, this would have had add
extra time, which we didn’t have, to the timelines. Not too much extra time, but every day
counted.
Not Sure Exactly the Naming for Each Component
We knew the capabilities that were needed, but we didn’t know the what each component should
be called

How Did We Fix It?

Technique 1 –Define the Capabilities
Campaign Send-out
Executor
ADAPTER
Marketing
Compliance
ADAPTER
Campaign Customer
Details
ADAPTER
Campaign Send-out
Snapshot
ADAPTER
Processing Customer
Selections
Producing Associations
Receiving Associated
Customer
Rules Execution
Checking External Opt-
outs
Checking Internal Opt-
outs
Customer Locker
Management
Requesting Comms
Details
Processing customer
selections
Snapshot Access
Local Snapshot
Management
Sending Communications

Technique 2 –Clear Packaging
01 Ensured that capabilities were self-evident
02 Components were clearly package too
03 Ports and Adapters model was clearly packaged
04 Consistency across the components

Technique 3 – Ports & Adapters
01 Clearly separated adapter layers – REST, messaging, etc
02 Repositories returned complete root aggregates
03 Extracted utility classes into a utils package
04 Surface the business logic in the domain package
05 Ensured adaptor layer naming doesn’t pollute domain

Technique 4 – Setup Topics
01 Topics created even though we could use a JAVA service call
02 JAVA service calls instead of REST
03 Gave great flexibility when extracting
Campaign
Send-out
Snapshot
ADAPTER
Campaign
Customer
Details
ADAPTER
Marketing
Compliance
ADAPTER
Campaign
Send-out
Executor
ADAPTER
JAVA
JAVA

Technique 5 – No DB Table Linking
01 Ensured no database foreign keys to other components
02 Didn’t need to create separate DB due to data short-lived
03 Repositories weren’t shared across context boundaries

Technique 6 – Bounded Context via Service
send-out execution local snapshot management snapshot management
LocalStorageService LocalStorageManagementService SegmentGroupExecitionService
fallbackRefreshLocalStorage findExecution
When we needed to move out the snapshot management, the
service call became a REST call but nothing on the send-out
execution side needed to change.
LocalStorageService LocalStorageManagementService SegmentGroupExecitionService
http://?
Campaign Send-out Executor Campaign Snapshot
fallbackRefreshLocalStorage

Lift and Shift
Marketing Compliance took two hours to extract
Campaign Snapshot took longer as it has Java service calls
Campaign Customer Details also has Java service calls
Feature switches used to transition between components
Some components did have some unwanted coupling

Keys to Success
01 We hit our commercial goal – just about
02 Core of architecture was sound - solid foundations for the future
03 Clear package names for the capabilities – domain evident
04 Ensured no violation of context boundaries – via service layer
05 Deferred some decisions for later, meaning we could focus

Things We Got Wrong/Pain Points
Some coupling with common code – mostly logging
Not all bounded contexts were correctly modelled
Root aggregate in Campaign Customer Details was wrong
It took a few discussions to get the feature switching correct

Software Principles and Project Deadlines Don't have to be Polar Opposites.pdf

Recommended

Recommended

More Related Content

Similar to Software Principles and Project Deadlines Don't have to be Polar Opposites.pdf

Similar to Software Principles and Project Deadlines Don't have to be Polar Opposites.pdf (20)

Recently uploaded

Recently uploaded (20)

Software Principles and Project Deadlines Don't have to be Polar Opposites.pdf