1.
@crichardson
Building microservices with
Scala, functional domain
models and Spring Boot
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com

2.
@crichardson
Presentation goal
Share my experiences with building an
application using Scala, functional domain
models, microservices, event sourcing,
CQRS, and Spring Boot

3.
@crichardson
About Chris

4.
@crichardson
About Chris
Founder of a buzzword compliant (stealthy, social, mobile,
big data, machine learning, ...) startup
Consultant helping organizations improve how they
architect and deploy applications using cloud, micro
services, polyglot applications, NoSQL, ...

5.
@crichardson
Agenda
Why build event-driven microservices?
Overview of event sourcing
Designing microservices with event sourcing
Implementing queries in an event sourced application
Building and deploying microservices

6.
@crichardson
Let’s imagine that you are
building a banking app...

7.
@crichardson
Domain model
Account
balance
open(initialBalance)
debit(amount)
credit(amount)
TransferTransaction
fromAccountId
toAccountId
amount

8.
@crichardson
Tomcat
Traditional application
architecture
Browser/
Client
WAR/EAR
RDBMS
Customers
Accounts
Transactions
BankingBanking UI
develop
test
deploy
Simple to
Load
balancer
scale
Spring MVC
Spring
Hibernate
...
HTML
REST/JSON

9.
@crichardson
Problem #1: monolithic
architecture
Intimidates developers
Obstacle to frequent deployments
Overloads your IDE and container
Obstacle to scaling development
Requires long-term commitment to a technology stack

10.
@crichardson
Solution #1: use a microservice
architecture
Banking UI
Account
Management Service
Transaction
Management Service
Account
Database
Transaction
Database
Standalone
services

11.
@crichardson
Problem #2: relational
databases
Scalability
Distribution
Schema updates
O/R impedance mismatch
Handling semi-structured data

12.
@crichardson
Solution #2: use NoSQL
databases
Avoids the limitations of RDBMS
For example,
text search Solr/Cloud Search
social (graph) data Neo4J
highly distributed/available database Cassandra
...

13.
@crichardson
But now we have problems
with data consistency!

14.
@crichardson
Problem #3: Microservices =
distributed data management
Each microservice has it’s own database
Some data is replicated and must be kept in sync
Business transactions must update data owned by
multiple services
Tricky to implement reliably without 2PC

15.
@crichardson
Problem #4: NoSQL =
ACID-free, denormalized databases
Limited transactions, i.e. no ACID transactions
Tricky to implement business transactions that update
multiple rows, e.g. http://bit.ly/mongo2pc
Limited querying capabilities
Requires denormalized/materialized views that must be
synchronized
Multiple datastores (e.g. DynamoDB + Cloud Search )
that need to be kept in sync

16.
@crichardson
Solution to #3/#4: Event-based
architecture to the rescue
Microservices publish events when state changes
Microservices subscribe to events
Synchronize replicated data
Maintains eventual consistency across multiple
aggregates (in multiple datastores)

17.
@crichardson
Eventually consistent money transfer
Message Bus
Transaction management
service
Account
management
service
transferMoney()
Publishes:
Subscribes to:
Subscribes to:
publishes:
TransferTransactionCreatedEvent
AccountDebitedEvent
DebitRecordedEvent
AccountCreditedEvent
TransferTransactionCreatedEvent
DebitRecordedEvent
AccountDebitedEvent
AccountCreditedEvent

18.
@crichardson
But reliably generating events
is difficult
Must atomically update datastore and publish event(s)
Non-option: datastore and message broker use 2PC
Use datastore as message queue
Local transaction updates state and publishes message
See BASE: An Acid Alternative, http://bit.ly/ebaybase
But
Business logic and event publishing code intertwined
Tricky to implement with aggregate-oriented NoSQL database

19.
@crichardson
Agenda
Why build event-driven microservices?
Overview of event sourcing
Designing microservices with event sourcing
Implementing queries in an event sourced application
Building and deploying microservices

20.
@crichardson
Event sourcing
For each aggregate:
Identify (state-changing) domain events
Define Event classes
For example,
Account: AccountOpenedEvent, AccountDebitedEvent,
AccountCreditedEvent
ShoppingCart: ItemAddedEvent, ItemRemovedEvent,
OrderPlacedEvent

21.
@crichardson
Persists events
NOT current state
Account
balance
open(initial)
debit(amount)
credit(amount)
AccountOpened
Event table
AccountCredited
AccountDebited
101 450
Account table
X
101
101
101
901
902
903
500
250
300

22.
@crichardson
Replay events to recreate
state
Account
balance
AccountOpenedEvent(balance)
AccountDebitedEvent(amount)
AccountCreditedEvent(amount)
Events

23.
@crichardson
Aggregate traits
Map Command to Events
Apply event returning
updated Aggregate

24.
@crichardson
Account - command processing
Prevent
overdraft

25.
@crichardson
Account - applying events
Immutable

26.
@crichardson
Request handling in an event-sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Account
new()
applyEvents(pastEvents)
newEvents = processCmd(SomeCmd)
saveEvents(newEvents)
Microservice A

27.
@crichardson
Event Store publishes events -
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
Aggregate
NoSQL
materialized
view
update()
update()
Microservice B

28.
@crichardson
Event Store API
trait EventStore {
def save[T <: Aggregate[T]](entity: T, events: Seq[Event],
assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]]
def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion,
entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]]
def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) :
Future[EntityWithIdAndVersion[T]]
def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId)
Future[Option[EntityWithIdAndVersion[T]]]
def subscribe(subscriptionId: SubscriptionId):
Future[AcknowledgableEventStream]
}
In case you are wondering:
Akka Persistence is too Akka-centric

29.
@crichardson
Benefits of event sourcing
Business:
Built in audit log
Enables temporal queries
Technical:
Solves data consistency issues in a Microservice/NoSQL-based architecture:
Atomically save and publish events
Event subscribers update other aggregates ensuring eventual consistency
Event subscribers update materialized views in SQL and NoSQL databases
Eliminates O/R mapping problem

30.
@crichardson
Drawbacks of event sourcing
Weird and unfamiliar
Events = a historical record of your bad design decisions
Handling duplicate events can be tricky
Idempotent commands
Duplicate detection, e.g. track most recently seen event

31.
@crichardson
Agenda
Why build event-driven microservices?
Overview of event sourcing
Designing microservices with event sourcing
Implementing queries in an event sourced application
Building and deploying microservices

32.
@crichardson
The anatomy of a microservice
Event Store
HTTP Request
HTTP Adapter
Aggregate
Event Adapter
Cmd
Cmd
Events
Events
Xyz Adapter
Xyz Request
microservice

33.
@crichardson
Asynchronous Spring MVC
controller

34.
@crichardson
AccountTransactionService
DSL concisely specifies:
1.Creates new TransferTransaction
2.Processes command
3.Applies events
4.Persists events

35.
@crichardson
TransferTransaction
Aggregate

36.
@crichardson
Handling events published
by Accounts
1.Load TransferTransaction
2.Processes command
3.Applies events
4.Persists events

37.
@crichardson
Agenda
Why build event-driven microservices?
Overview of event sourcing
Designing microservices with event sourcing
Implementing queries in an event sourced application
Building and deploying microservices

38.
@crichardson
Let’s imagine that you want
to display an account and it’s
recent transactions...

39.
@crichardson
Displaying balance + recent
transactions
We need to do a “join: between the Account and the corresponding
TransferTransactions
(Assuming Debit/Credit events don’t include other account, ...)
BUT
Event Store = primary key lookup of individual aggregates, ...
Use Command Query Responsibility Separation
Define separate “materialized” query-side views that implement
those queries

40.
@crichardson
Query-side microservices
Event Store
Updater - microservice
View Updater
Service
Events
Reader - microservice
HTTP GET
Request
View Query
Service
View
Store
e.g.
MongoDB
Neo4J
CloudSearch
update query

41.
@crichardson
Persisting account balance and
recent transactions in MongoDB
{
id: "298993498",
balance: 100000,
transactions : [
{"transactionId" : "4552840948484",
"fromAccountId" : 298993498,
"toAccountId" : 3483948934,
"amount" : 5000}, ...
],
changes: [
{"changeId" : "93843948934",
"transactionId" : "4552840948484",
"transactionType" : "AccountDebited",
"amount" : 5000}, ...
]
}
Denormalized = efficient lookup
Transactions that
update the account
The sequence of
debits and credits

42.
@crichardson
Updating MongoDB using
Spring Data
class AccountInfoUpdateService
(mongoTemplate : MongoTemplate, ...)
extends CompoundEventHandler {
@EventHandler
def recordDebit(de: DispatchedEvent[AccountDebitedEvent]) = {
...
val ci = AccountChangeInfo(...)
mongoTemplate.updateMulti(
new Query(where("id").is(de.entityId.id).and("version").lt(changeId)),
new Update().
dec("balance", amount).
push("changes", ci).
set("version", changeId),
classOf[AccountInfo])
}
@EventHandler
def recordTransfer(de: DispatchedEvent[TransferTransactionCreatedEvent]) = ...
}
insert/In-place update
duplicate event detection
updates to and from accounts

43.
@crichardson
Retrieving account info from
MongoDB using Spring Data
class AccountInfoQueryService(accountInfoRepository : AccountInfoRepository)
{
def findByAccountId(accountId : EntityId) : AccountInfo =
accountInfoRepository.findOne(accountId.id)
}
case class AccountInfo(id : String, balance : Long,
transactions : List[AccountTransactionInfo],
changes : List[ChangeInfo],
version : String)
case class AccountTransactionInfo(changeId : String,
transactionId : String,
transactionType : String,
amount : Long, balanceDelta : Long)
trait AccountInfoRepository extends MongoRepository[AccountInfo, String]
Implementation generated by Spring Data

44.
@crichardson
Agenda
Why build event-driven microservices?
Overview of event sourcing
Designing microservices with event sourcing
Implementing queries in an event sourced application
Building and deploying microservices

45.
@crichardson
Building microservices with
Spring Boot
Makes it easy to create stand-alone, production ready
Spring Applications
Automatically configures Spring using Convention over
Configuration
Externalizes configuration
Generates standalone executable JARs with embedded
web server

46.
@crichardson
Spring Boot simplifies configuration
Spring
Container
Application
components
Fully
configured
application
Configuration
Metadata
•Typesafe JavaConfig
•Annotations
•Legacy XML
Default
Configuration
Metadata
Spring
Boot
You write less
of this
Inferred from
CLASSPATH

47.
@crichardson
Tiny Spring configuration for
Account microservice
@Configuration
@EnableAutoConfiguration
@Import(classOf[JdbcEventStoreConfiguration]))
@ComponentScan
class AccountConfiguration {
@Bean
def accountService(eventStore : EventStore) =
new AccountService(eventStore)
@Bean
def accountEventHandlers(eventStore : EventStore) =
EventHandlerRegistrar.makeFromCompoundEventHandler(
eventStore, "accountEventHandlers",
new TransferWorkflowAccountHandlers(eventStore))
@Bean
@Primary
def scalaObjectMapper() = ScalaObjectMapper
}
Service
Event handlers
Scan for controllers
Customize JSON serialization

48.
@crichardson
The Main program
object BankingMain extends App {
SpringApplication.run(classOf[AccountConfiguration], args :_ *)
}

49.
@crichardson
Building with Gradle
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.0.0.RELEASE")
}
}
apply plugin: 'scala'
apply plugin: 'spring-boot'
...

50.
@crichardson
Running the microservice
$ java -jar build/libs/banking-main-1.0-SNAPSHOT.jar --server.port=8081
...
11:38:04.633 INFO n.c.e.e.bank.web.main.BankingMain$ - Started
BankingMain. in 8.811 seconds (JVM running for 9.884)
$ curl localhost:8081/health
{"status":"UP",
"mongo":{"status":"UP","version":"2.4.10"},
"db":{"status":"UP","database":"H2","hello":1}
}
Built in health checks
Command line arg processing

51.
@crichardson
Jenkins-based deployment
pipeline
Build & Test
micro-
service
Build & Test
Docker
image
Deploy
Docker
image
to
Repository
One pipeline per micro-service

52.
@crichardson
Summary
Event Sourcing solves key data consistency issues with:
Microservices
Partitioned/NoSQL databases
Spring and Scala play nicely together
Spring Boot makes it very easily to build production ready
microservices

53.
@crichardson
Questions?
@crichardson chris@chrisrichardson.net
http://plainoldobjects.com