Building microservices with Scala, functional domain models and Spring Boot

37,518 views

Published on

In this talk you will learn about a modern way of designing applications that’s very different from the traditional approach of building monolithic applications that persist mutable domain objects in a relational database.We will talk about the microservice architecture, it’s benefits and drawbacks and how Spring Boot can help. You will learn about implementing business logic using functional, immutable domain models written in Scala. We will describe event sourcing and how it’s an extremely useful persistence mechanism for persisting functional domain objects in a microservices architecture.

Published in: Software
3 Comments
92 Likes
Statistics
Notes
  • @Santanu Dey I talk about the tradeoffs in other presentations and in this article: http://www.infoq.com/articles/microservices-intro

    Generally speaking large applications should consist of a set of microservices. IMHO it's development and deployment is too painful otherwise. Of course, the definition of 'large' is up for debate :-)

    For example, what's great about a microservice architecture is that if, for example, a change only impacts a single service then it's trivial to implement, test and deploy that change. Where some of the pain comes in is if you have changes that span multiple services.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @ Chris Richardson, micro-services is inherently complicated. My personal assessment is that it is 10x harder to debug, manage and build micro-services applications compared to traditional ones. Are the benefits large enough to move out from large monolithic application architecture? Appreciate your views on this.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Very consistent presentation.
    Way beyond hello-world stuff.
    That's what I call a modern web app.
    http://plainoldobjects.com/ is worth checking out.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
37,518
On SlideShare
0
From Embeds
0
Number of Embeds
15,392
Actions
Shares
0
Downloads
618
Comments
3
Likes
92
Embeds 0
No embeds

No notes for slide

Building microservices with Scala, functional domain models and Spring Boot

  1. 1. @crichardson Building microservices with Scala, functional domain models and Spring Boot Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com
  2. 2. @crichardson Presentation goal Share my experiences with building an application using Scala, functional domain models, microservices, event sourcing, CQRS, and Spring Boot
  3. 3. @crichardson About Chris
  4. 4. @crichardson About Chris Founder of a buzzword compliant (stealthy, social, mobile, big data, machine learning, ...) startup Consultant helping organizations improve how they architect and deploy applications using cloud, micro services, polyglot applications, NoSQL, ...
  5. 5. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  6. 6. @crichardson Let’s imagine that you are building a banking app...
  7. 7. @crichardson Domain model Account balance open(initialBalance) debit(amount) credit(amount) TransferTransaction fromAccountId toAccountId amount
  8. 8. @crichardson Tomcat Traditional application architecture Browser/ Client WAR/EAR RDBMS Customers Accounts Transactions BankingBanking UI develop test deploy Simple to Load balancer scale Spring MVC Spring Hibernate ... HTML REST/JSON
  9. 9. @crichardson Problem #1: monolithic architecture Intimidates developers Obstacle to frequent deployments Overloads your IDE and container Obstacle to scaling development Requires long-term commitment to a technology stack
  10. 10. @crichardson Solution #1: use a microservice architecture Banking UI Account Management Service Transaction Management Service Account Database Transaction Database Standalone services
  11. 11. @crichardson Problem #2: relational databases Scalability Distribution Schema updates O/R impedance mismatch Handling semi-structured data
  12. 12. @crichardson Solution #2: use NoSQL databases Avoids the limitations of RDBMS For example, text search Solr/Cloud Search social (graph) data Neo4J highly distributed/available database Cassandra ...
  13. 13. @crichardson But now we have problems with data consistency!
  14. 14. @crichardson Problem #3: Microservices = distributed data management Each microservice has it’s own database Some data is replicated and must be kept in sync Business transactions must update data owned by multiple services Tricky to implement reliably without 2PC
  15. 15. @crichardson Problem #4: NoSQL = ACID-free, denormalized databases Limited transactions, i.e. no ACID transactions Tricky to implement business transactions that update multiple rows, e.g. http://bit.ly/mongo2pc Limited querying capabilities Requires denormalized/materialized views that must be synchronized Multiple datastores (e.g. DynamoDB + Cloud Search ) that need to be kept in sync
  16. 16. @crichardson Solution to #3/#4: Event-based architecture to the rescue Microservices publish events when state changes Microservices subscribe to events Synchronize replicated data Maintains eventual consistency across multiple aggregates (in multiple datastores)
  17. 17. @crichardson Eventually consistent money transfer Message Bus Transaction management service Account management service transferMoney() Publishes: Subscribes to: Subscribes to: publishes: TransferTransactionCreatedEvent AccountDebitedEvent DebitRecordedEvent AccountCreditedEvent TransferTransactionCreatedEvent DebitRecordedEvent AccountDebitedEvent AccountCreditedEvent
  18. 18. @crichardson But reliably generating events is difficult Must atomically update datastore and publish event(s) Non-option: datastore and message broker use 2PC Use datastore as message queue Local transaction updates state and publishes message See BASE: An Acid Alternative, http://bit.ly/ebaybase But Business logic and event publishing code intertwined Tricky to implement with aggregate-oriented NoSQL database
  19. 19. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  20. 20. @crichardson Event sourcing For each aggregate: Identify (state-changing) domain events Define Event classes For example, Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEvent ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent
  21. 21. @crichardson Persists events NOT current state Account balance open(initial) debit(amount) credit(amount) AccountOpened Event table AccountCredited AccountDebited 101 450 Account table X 101 101 101 901 902 903 500 250 300
  22. 22. @crichardson Replay events to recreate state Account balance AccountOpenedEvent(balance) AccountDebitedEvent(amount) AccountCreditedEvent(amount) Events
  23. 23. @crichardson Aggregate traits Map Command to Events Apply event returning updated Aggregate
  24. 24. @crichardson Account - command processing Prevent overdraft
  25. 25. @crichardson Account - applying events Immutable
  26. 26. @crichardson Request handling in an event-sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Account new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A
  27. 27. @crichardson Event Store publishes events - consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate NoSQL materialized view update() update() Microservice B
  28. 28. @crichardson Event Store API trait EventStore { def save[T <: Aggregate[T]](entity: T, events: Seq[Event], assignedId : Option[EntityId] = None): Future[EntityWithIdAndVersion[T]] def update[T <: Aggregate[T]](entityIdAndVersion : EntityIdAndVersion, entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]] def find[T <: Aggregate[T] : ClassTag](entityId: EntityId) : Future[EntityWithIdAndVersion[T]] def findOptional[T <: Aggregate[T] : ClassTag](entityId: EntityId) Future[Option[EntityWithIdAndVersion[T]]] def subscribe(subscriptionId: SubscriptionId): Future[AcknowledgableEventStream] } In case you are wondering: Akka Persistence is too Akka-centric
  29. 29. @crichardson Benefits of event sourcing Business: Built in audit log Enables temporal queries Technical: Solves data consistency issues in a Microservice/NoSQL-based architecture: Atomically save and publish events Event subscribers update other aggregates ensuring eventual consistency Event subscribers update materialized views in SQL and NoSQL databases Eliminates O/R mapping problem
  30. 30. @crichardson Drawbacks of event sourcing Weird and unfamiliar Events = a historical record of your bad design decisions Handling duplicate events can be tricky Idempotent commands Duplicate detection, e.g. track most recently seen event
  31. 31. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  32. 32. @crichardson The anatomy of a microservice Event Store HTTP Request HTTP Adapter Aggregate Event Adapter Cmd Cmd Events Events Xyz Adapter Xyz Request microservice
  33. 33. @crichardson Asynchronous Spring MVC controller
  34. 34. @crichardson AccountTransactionService DSL concisely specifies: 1.Creates new TransferTransaction 2.Processes command 3.Applies events 4.Persists events
  35. 35. @crichardson TransferTransaction Aggregate
  36. 36. @crichardson Handling events published by Accounts 1.Load TransferTransaction 2.Processes command 3.Applies events 4.Persists events
  37. 37. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  38. 38. @crichardson Let’s imagine that you want to display an account and it’s recent transactions...
  39. 39. @crichardson Displaying balance + recent transactions We need to do a “join: between the Account and the corresponding TransferTransactions (Assuming Debit/Credit events don’t include other account, ...) BUT Event Store = primary key lookup of individual aggregates, ... Use Command Query Responsibility Separation Define separate “materialized” query-side views that implement those queries
  40. 40. @crichardson Query-side microservices Event Store Updater - microservice View Updater Service Events Reader - microservice HTTP GET Request View Query Service View Store e.g. MongoDB Neo4J CloudSearch update query
  41. 41. @crichardson Persisting account balance and recent transactions in MongoDB { id: "298993498", balance: 100000, transactions : [ {"transactionId" : "4552840948484", "fromAccountId" : 298993498, "toAccountId" : 3483948934, "amount" : 5000}, ... ], changes: [ {"changeId" : "93843948934", "transactionId" : "4552840948484", "transactionType" : "AccountDebited", "amount" : 5000}, ... ] } Denormalized = efficient lookup Transactions that update the account The sequence of debits and credits
  42. 42. @crichardson Updating MongoDB using Spring Data class AccountInfoUpdateService (mongoTemplate : MongoTemplate, ...) extends CompoundEventHandler { @EventHandler def recordDebit(de: DispatchedEvent[AccountDebitedEvent]) = { ... val ci = AccountChangeInfo(...) mongoTemplate.updateMulti( new Query(where("id").is(de.entityId.id).and("version").lt(changeId)), new Update(). dec("balance", amount). push("changes", ci). set("version", changeId), classOf[AccountInfo]) } @EventHandler def recordTransfer(de: DispatchedEvent[TransferTransactionCreatedEvent]) = ... } insert/In-place update duplicate event detection updates to and from accounts
  43. 43. @crichardson Retrieving account info from MongoDB using Spring Data class AccountInfoQueryService(accountInfoRepository : AccountInfoRepository) { def findByAccountId(accountId : EntityId) : AccountInfo = accountInfoRepository.findOne(accountId.id) } case class AccountInfo(id : String, balance : Long, transactions : List[AccountTransactionInfo], changes : List[ChangeInfo], version : String) case class AccountTransactionInfo(changeId : String, transactionId : String, transactionType : String, amount : Long, balanceDelta : Long) trait AccountInfoRepository extends MongoRepository[AccountInfo, String] Implementation generated by Spring Data
  44. 44. @crichardson Agenda Why build event-driven microservices? Overview of event sourcing Designing microservices with event sourcing Implementing queries in an event sourced application Building and deploying microservices
  45. 45. @crichardson Building microservices with Spring Boot Makes it easy to create stand-alone, production ready Spring Applications Automatically configures Spring using Convention over Configuration Externalizes configuration Generates standalone executable JARs with embedded web server
  46. 46. @crichardson Spring Boot simplifies configuration Spring Container Application components Fully configured application Configuration Metadata •Typesafe JavaConfig •Annotations •Legacy XML Default Configuration Metadata Spring Boot You write less of this Inferred from CLASSPATH
  47. 47. @crichardson Tiny Spring configuration for Account microservice @Configuration @EnableAutoConfiguration @Import(classOf[JdbcEventStoreConfiguration])) @ComponentScan class AccountConfiguration { @Bean def accountService(eventStore : EventStore) = new AccountService(eventStore) @Bean def accountEventHandlers(eventStore : EventStore) = EventHandlerRegistrar.makeFromCompoundEventHandler( eventStore, "accountEventHandlers", new TransferWorkflowAccountHandlers(eventStore)) @Bean @Primary def scalaObjectMapper() = ScalaObjectMapper } Service Event handlers Scan for controllers Customize JSON serialization
  48. 48. @crichardson The Main program object BankingMain extends App { SpringApplication.run(classOf[AccountConfiguration], args :_ *) }
  49. 49. @crichardson Building with Gradle buildscript { repositories { mavenCentral() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:1.0.0.RELEASE") } } apply plugin: 'scala' apply plugin: 'spring-boot' ...
  50. 50. @crichardson Running the microservice $ java -jar build/libs/banking-main-1.0-SNAPSHOT.jar --server.port=8081 ... 11:38:04.633 INFO n.c.e.e.bank.web.main.BankingMain$ - Started BankingMain. in 8.811 seconds (JVM running for 9.884) $ curl localhost:8081/health {"status":"UP", "mongo":{"status":"UP","version":"2.4.10"}, "db":{"status":"UP","database":"H2","hello":1} } Built in health checks Command line arg processing
  51. 51. @crichardson Jenkins-based deployment pipeline Build & Test micro- service Build & Test Docker image Deploy Docker image to Repository One pipeline per micro-service
  52. 52. @crichardson Summary Event Sourcing solves key data consistency issues with: Microservices Partitioned/NoSQL databases Spring and Scala play nicely together Spring Boot makes it very easily to build production ready microservices
  53. 53. @crichardson Questions? @crichardson chris@chrisrichardson.net http://plainoldobjects.com

×