1.
Jeppe Cramon – Partner in TigerTeam ApS
AANUG Conference
September 2013
SOA & EDA

2.
There are problems with SOA
• Systems are more fragile
• Development and Maintenance costs are higher
• Your services are not being reused
• You thought SOA would solve your integration
problems, but they’ve gotten worse
• No one wants to build or maintain services
• System performance is worse

3.
If you have arrived at the realization
that SOA is a pain and costs too
much without enabling business
agility or reducing your costs, you
are not the only one.
You will have to change your
approach to achieve the benefits of
SOA

4.
Let’s take a trip back in time

5.
And look at how we could do business

6.
Sales Department
Old style SOA
Design Department Customer Department Finance Department Fabric Department
The Customer
Create Sales Order

7.
This is slow

8.
We are not selling enough!!

9.
Sales Department
Old style SOA II
Design Department Customer Department Finance Department Fabric Department
Create Sales Order

10.
We need to speed things up!
Hackel walks between desks?
Way too slow.

11.
We need to speed things up!
He needs to call So we make sure that
he can
Actually, call a lot!

12.
We need to speed things up!
But Hackel still have to wait Even if we get creative

13.
We need to speed things up!
Something
needs to be
done!!

14.
Speed up even more!
Lets get an automatic switchboard!

15.
Speed up even more!
New technology, same
problem
Even if we get creative
again

16.
Or do things Async
So Hackel can send request
to Ingeborg and Kresten
which can respond to
Hackel when they’ve got an
answer
But even that doesn’t
scale well, there’s still only
one Ingeborg and Kresten

17.
Speed up even more!
So Kresten and Ingeborg adds more staff

18.
Scalability is expensive
Sales Department
Design Department Customer Department Finance Department Fabric Department
Coordinator Hackel

19.
Same problem different technology
Data
Storage
Data
Storage
Data
Storage
Finance
Service
Data
Service
Design
Service
Hackel Coordinator
Service
Hackel Coordinator
Service
Sales Order
Service
Sales Order
Service
Customer Customer
Fabric
Service
Customer
Service
Data
Storage
Client
Process Service layer
Activity Service layer
Data Service layer

20.
Resemble much?

21.
Now back to the story…
• Scaling up Sales requires scaling up other
departments relatively much
• People cost money and why have 9 Finance
people for every 10 sales people just to
answer questions about pricing
• There’s got to be a better way to handle this
• And there is 

22.
Sales Department
Event Driven SOA and local caching
Design Department Customer Department Finance Department Fabric Department
The Customer
Create Sales Order

23.
Scalability is cheaper
Sales Department
Design Department Customer Department Finance Department Fabric Department

24.
This is just the beginning

25.
Have our mindset changed over
the last 40 years?

26.
Paper work, file cabinets and manual workflows
Inventory ShippingPurchase
Manual workflow Manual workflow

27.
Next step was automating paper forms
(aka. The birth of CRUD)
Inventory ShippingPurchase
Manual workflow Manual workflow

28.
Database integration
Inventory ShippingPurchase
Database
Integration
Database
Integration

29.
Enterprise Application Integration (EAI)
Inventory ShippingPurchase
Application
Integration
Application
Integration

30.
WebServices to the Rescue?
SOA Web of Mess
Inventory
Purchase
ShippingPortal

31.
ESB’s the the Rescue?
SOA Web of Mess hidden by ESB
Inventory
Purchase
ShippingPortal
ESB

32.
ESB Reality
SOA Web of Mess hidden by ESB
Inventory
Purchase
ShippingPortal
ESB

33.
I’ve got a cheaper solution

34.
To be clear – none of the examples
represent in my opinion SOA
The 4 tenets of SOA
1. Services are autonomous
– Encapsulation & Cohesion at a bigger scale.
– A service is autonomous if it doesn’t rely on other services to complete its job
2. Services have explicit boundaries
– Services should not be too familiar with each other. There’s a clear boundary
to where the responsibilities of each services starts and ends
– Boundaries are made unclear when a service rely on other services to
complete its job
3. Services share contract and schema, not class or type
– Encapsulation improved. We need not worry about how things are
implemented (languages or platform dependent information)
4. Service interaction is controlled by a policy
– Controls under which rules/form of technical communication between
services take place (e.g. using encryption incl. algorithm and keys)

35.
What we’ve seen is just old EAI
wine on new SOA bottles

36.
WebServices and in general
synchronous integration has nothing
to do with real SOA
Some call this pattern for SOA 1.0 to
distinguish them selves from the old
approach and the mess it causes

37.
Layered Architectures typically leaves all
orchestration to a central manager (ESB)
where business processes are coordinated
through spaghetti code (BPEL)

38.
These BPEL processes typically break
all Service encapsulation as they are
data and feature envious
This hurts our coupling an autonomy
even further

39.
What we have with classic layered
SOA is a HARD COUPLED
architecture

40.
But what about reuse?
• Layered SOA has typically been recommended because it increases reuse
(at the expense of a hard coupled architecture)
• The thesis is that the more we reuse, the faster we will be done
• But the thesis rest on the false assumption that writing code is the most
expensive part of a project
– Not all code takes the same amount of time to write. Some code is very trivial
while other code is very hard to write
• The real money and time consumers on any project are:
– Figuring out what the customer needs
– The time it takes for the customer to figure what they really needed and the
resulting rework
– Meetings
– Integration work, databases, webservices
– Fix bugs
– Debugging
– Deployment
– Test
– Ship
Faile
d?
Yes
No
The more dependencies
we have, the worse and
more expensive this
becomes

41.
Service reuse multiplies our direct
and especially indirect dependencies
which creates high coupling
My new
Service that
wants to reuse
other services
Service that is
going to be
reused
Reusable
Service
Reusable
Service
Reusable
Service
Reusable
Service
Reusable
Service
DB
service
Another
Service that is
going to be
reused

42.
Can’t we just avoid Services and
make one big application?

43.
Retail Domain
Invoicing
Subdomain
Product
Catalog
Subdomain
Shipping
Subdomain
Sales (Order)
Subdomain
Inventory
Subdomain
Pricing

44.
One Domain Model to Rule them All?
Combining subdomains into one big
domain model increases complexity as
everything tends to be connected to
and depend on everything else

45.
One big model to capture it ALL?
….
…..
…..
….
…..
…..
….
…..
…..
….
…..
…..

46.
Result: Our queries get messy

47.
ONE MODEL TO RULE THEM ALL
ONE MODEL TO FIND THEM
ONE MODEL TO BRING THEM ALL
AND IN THE DARKNESS BIND THEM

48.
The database determines our scalability

49.
And databases often scale poorly
To solve the performance
problems for a few, you
have to upgrade it all

50.
Do not design for locality and try to
distribute
Design for distribution, take advantage
of locality

51.
Alternatively we can use Read replicas
Master Slave SlaveSlaveSlave
You’re now eventually consistent

52.
Or introduce caching…
What’s your synchronization mechanism?
?

53.
Many perspectives on data
Customer
Retail System
Pricing
Inventory
Sales
Product
Unit Price
Promotional Price
Promotion End Date
Stock Keeping Unit (SKU)
Quantity On Hand (QOH)
Location Code
Price
Quantity Ordered
Name
The lifecycle for Data is VERY important!
Forecasting?

54.
Smaller models & clear data ownership
Sales
Product
ProductID
Name
Description
Quantity Ordered
…
Sales
Inventory
Product
ProductID
SKU
QOH
Location Code
… Inventory
Pricing
Product
ProductID
Unit Price
Promotional Price
…
Pricing
DDD:
Bounded
Context
SOA:
Service
Retail System
Shared Entity identity

55.
A Service is
• A technical authority for a specific business
capability – very similar to Bounded Contexts
• The owner of all the data and business rules
that support this business capability – like
Bounded Contexts
A Service is equivalent to a
Bounded Context

56.
What a Service is NOT
• A service with only functionality (and no data) is a
FUNCTION
– Like: check if order is valid
• A service that only has data is a DATABASE
– Like: CRUD of an entity
This is typically seen in a lot of layered SOA usages
where a function calls a function that calls a
function that calls a database

57.
Autonomy becomes the next
(and biggest) problem
Can you rely on others?

58.
Data Layer
Service Orchestration issue
Data Access Layer
Service Layer
Interface/Display
Layer
System CSystem BSystem A
UI
Service Service
DAL DAL DAL
Schema Schema Schema
Local transaction
spanning all
modules
Service

59.
Service autonomy
Service B Service C
Service A
System X
Service A
System Y
Service B Service C
System X
Slow/unreliable network
Different SLA
Slow system

60.
Orchestration using synchronous
communication and local transactions
System CSystem BSystem A
UI
Service Service
DAL DAL DAL
Schema Schema Schema
B:Service()
call C:Service()
call B:DAL()
call A:Service()
commit()
Service

61.
Solution for when going distributed?
Can’t we just distributed transactions?
(XA / 2 phase commit)

62.
Distributed Transactions…. Oh my
Sales system
Sale
Delivery
system Deliveries
Customer/CR
M system Custome
r
SAP Bookkeeping
Complete
Purchase
Transaction
Coordinator
Transactional
Resource
Prepare
Phase
Commit
Phase
2 Phase Commit

63.
What’s wrong with distributed transactions?
• Transactions lock resources while active
• Services are autonomous
– Can’t be expected to finish within a certain time
interval
• Locking keeps other transactions from
completing their job
• Locking doesn’t scale
• X Phase Commit is fragile by design

64.
Orchestration using synchronous
communication and without distributed transactions
System CSystem BSystem A
UI
Service Service
DAL DAL DAL
Schema Schema Schema
ServiceB:Service()
call C:Service()
call B:DAL()
call A:Service()
if (A:Call-Failed?)
call A:Service()
if (A:Call-Failed?)
Wait-A-While()
call A:Service()
if (A:Call-Failed-Due-To-IO?)
Save-We-Need-Check-If-Call-A-Succeded-After-All
AND We-Need-To-Retry call C:Service and call B:DAL
AND Tell-Customer-That-This-Operation-Perhaps-Went-Well
if (A:Call-Went-Well)
commit()

65.
More of the same clearly can’t be
The answer…

66.
The more autonomous services are, the
more loosely coupled they are.
Loose coupling is what drives business
agility – which is why we wanted SOA in the
first place

67.
Loose coupling
Requires good encapsulation and
high cohesion

68.
Return of the SILOS!!!!

69.
We need SMALL SILOS that
SHARE very LITTLE with each other
Sales Pricing InventoryShipping
✕ ✕ ✕

70.
A silo/bounded-context contains all parts related to it
DB Schema
Domain
Services
Application
Services
Domain
Model
Aggregates
, Entities,
Value
Objects,
Events
Integration
Endpoints
(REST,
SOAP,
Pub/Sub)
User
Interface
Silo

71.
A silos parts are not limited to deployment
at a single location
Subdomain
User
Interface
Integration
Endpoints
(REST,
SOAP,
Pub/Sub)
Domain
Services
Application
Services
Aggregates,
Entities,
Value
Objects,
Events
DB
Schema
Units of individual deployment

72.
What should our silos share?
Bounded
Context

73.
Degrees of coupling
UI
UI
Service
ServiceData
Data Events
Events

74.
Composite UI’s
Retail System
Sales Pricing Inventory
Web/Application
tier
Background server
tier
Storage tier
Composite UI
UI Layered
ApplicationData Access Layer
Readmodel
Writemodel
✕ ✕

75.
Composite UI - example
Context:
Book
ISBN-10 0-321-12521-5

76.
Composite UI - HTML
#DIV – BookReviews
#DIV – BookImage
#DIV – BookTitleAndAuthor
#DIV – BookPricing
#DIV – BookAvailability
#DIV - Header
#DIV - Body

77.
Composite UI

78.
The Service owns it UI in all Contexts
and for all Composite UI’s
Not just for HTML clients

79.
Invoice Composite UI example
InvoiceHeader
Order:ShippingI
nfo
Invoice:
NextInvoiceNumb
er
Invoice:
Data and Due
date
Order:
RelationInformation
Order:Item-
Qty
Product:Ite
m
Product:
Description
Order:
Item-Unit-Price
Order:
Item-
Total-
Price
Order:Total
Billing:Balance
All Services
participate at
the UI level for
each individual
Item in the
Order

80.
When won’t a Composite UI make sense?
For certain types of UI’s that built outside of the
organization, e.g. customer specific netbank apps and
mobile banking application we need to provide a
synchronous integration API that will allow them to
integrate with the business domains without having to
coordinate with the business domains UI team.
The integration API could be built using both a REST*
and/or a SOAP approach.
* It’s not such a distant thought to see a Composite UI part as a REST resource
with a rendering matching the requested MimeType encoding.

81.
That covers data presentation
What about transactions?

82.
Next Step – Event Driven Architecture

83.
Why Events?

84.
Synchronous calls lowers our
Fault tolerance
• When servers crashes
• When databases are down
• When deadlocks occurs in the database
– Do you retry?
With synchronous RPC style Services interaction
we loose business data. There’s no automatic
retry and idempotence is often an after though.

85.
Cure: Messaging
• Message based communication is asynchronous
and thereby breaks temporal coupling
• Messages can be exchanged between services
over a message channel
• The Message channel is responsible for
delivering the message(s) to the relevant parties
(consumers). If something goes wrong, the
message will be put back on the Message channel
(rollback) and will be resent later

86.
Let’s make the implicit explicit!
Old wisdom seems to have been forgotten. Let’s introduce:
Business Events
Which:
• Signal that something has happened
• Closely aligned to the Domain Model
• Are handled by a messaging system
• They are in the past tense:
– CustomerBilled
– ParcelShipped
– CustomerCreated
– ReviewCreated
– CommentAdded
– CommentDeleted

87.
Business Events as Messages
A Business Event can be transferred over a
message channel, such as a Queue or a Topic, in
many different formats

88.
Business Event as XML Message
<OrderWasAccepted>
<CustomerId>50D1F244-ABBC-4EC7-BDCA-E4934C124A89</CustomerId>
<OrderId>C199322A-01F1-4E56-918E-7A63529F8FA3</OrderId>
<ShippingAddress> ... </ShippingAddress>
<BillingAddress> ... </BillingAddress>
<Items>
<Item ProductId="4CD22C4B-600C-4477-B5BF-48ABDEE4DA61" Amount="100"
AmountUnit="Pieces" UnitPrice="100,10" UnitCurrency="EUR"/>
<Item ProductId="56E6BD19-660C-464A-9120-100DAF579855" Amount="10"
AmountUnit="Litres" UnitPrice="56,95" UnitCurrency="CHF"/>
</Items>
</OrderWasAccepted>

89.
Business Event as JSON Message
{
EventType: "OrderWasAccepted",
CustomerId: "50D1F244-ABBC-4EC7-BDCA-E4934C124A89",
OrderId: "C199322A-01F1-4E56-918E-7A63529F8FA3",
ShippingAddress: { ... }
BillingAddress: { ... }
Items: [
{
ProductId: "4CD22C4B-600C-4477-B5BF-48ABDEE4DA61",
Amount: "100",
AmountUnit: "Pieces",
UnitPrice: "100,10",
UnitCurrency: "EUR"
},
{
ProductId: "56E6BD19-660C-464A-9120-100DAF579855",
Amount: "10",
AmountUnit: "Litres",
UnitPrice: "56,95",
UnitCurrency: "CHF"
}
]
}

90.
Business Events help us achieve autonomy,
loose coupling and encapsulation
• Encapsulation - because we don’t need to
supply our services internal data (because no
on else needs to know them) – internal data
will only be accessible through the Services UI
(which takes part in a Composite UI)
• Events only need to explain WHAT happened
and what it relates to and very few business
data (which are typically only needed for
replication scenarios)

91.
Business Events Messages
and
Business Processes
By publishing Events messages from our
Services we can communicate with each
other and also drive Business Processes

92.
Using Business Events to drive Business
Processes
Sales Service
Shipping
Invoicing
Sales
Customers
MessageChannel
The sales
fulfillment
processing
can now
begin…
Online Ordering System
Web Shop
(Composite UI)
Invoicing Service
Shipping Service
Order
Accepted
Event

93.
Business Events example
Sales Service
Order
Accepted
Invoicing Service
Retail System
Order
Accepted
Customer
Billed
MessageChannel
We use the Order Accepted event message published from the Sales Service
to drive the Invoicing/Billing of the customer.
The billing part of the process also use Business Events, in this case Customer Billed
Event, to indicate that its part of the process is completed.
Because we use asynchronous messaging we can still accept orders in the sales service
even though the invoicing services is down. The Order Accepted event message will
remain in the Message Channel until the Invoicing Service is ready to process it.

94.
Introducing Choreographed Processes
Sales Service
Order
Accepted
Invoicing Service
Shipping
Process Manager
(Saga)
Shipping Service
Retail System
Message
Channel
Order
Accepted
Order
Accepted
Customer
Billed
Customer
Billed
Order
Ready for
Shipment
Order
Ready for
Shipment
Works as a Finite
State Machine
(WorkFlow)
handling the life
cycle of Shipping
and thereby forms
a very central new
Aggregate in the
System

95.
Process Managers
• Process Managers are essential to the coordination and
monitoring of long running processes.
• They work as a Finite State Machines (WorkFlow)
which handling the life cycle of Process (e.g. Shipping
an Order) and thereby forms a very central new
Aggregate in the System
• They can include manual steps/person intervention
• Sometimes these Process Managers belong naturally
within a specific Subdomain and other times they are
truly and thing by themselves and often form new
Subdomains that way
Companies often derive their competitive advantages from their Processes.
A Process Manager allows you coordinate Business Processes on the basis of
Events

96.
Domain Events can also be used
for data replication
This is typically used for read-mostly master
data that is needed in many business-
domains. This way we avoid having to call a
synchronous services to query the latest
value(s)

97.
Event Based Replication
Subdomain Z
Repositor
y
Repositor
y
Z Data
Subdomain Y
Service Repository
Message channel
Event Receiver
Y Event Receiver and Local Cache Builder
Y Data
Cache
Y Events
Y Events

98.
Questions?
@jeppec on Twitterjeppe@tigerteam.dk
@TigerTeamDK on Twitterhttp://tigerteam.dk