Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Decomposing the Monolith using modern-day .NET and a touch of microservices

624 views

Published on

If I have to name a single biggest hype in software architecture land then it would be "microservice". They are supposed to be small and focused, can be deployed independently, can work with any technology and will solve all your monolithical problems. But we all know that silver bullets don't exist, plus technology should never be a goal, but merely a means to an end. Nonetheless, following the path towards real microservices is a great strategy for decomposing a monolith without the deployment complexity of the first. So how do you do that? What technologies does the .NET realm offer for us? In this talk, I'll show you some of the pros and cons of micro-services and its ingredients to leverage modern-day .NET and Event Sourcing to move your monolith into a bright new future.

Published in: Technology
  • Be the first to comment

Decomposing the Monolith using modern-day .NET and a touch of microservices

  1. 1. Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver
  2. 2. @ddoomen
  3. 3. Hybrid of Technologies Multiple patterns for the same problems Long compile time Large source control repositories Long-running unit tests Too much coupling Limited team scalability Devs need to know everything NoIsolation.Productivity drops over time. Proof of success!
  4. 4. Application Functional Module Functional Module Functional Module Functional Module Composition Each module owns/exposes its master data Does not depend on anything except for curated shared APIs Owns schema and storage (may use shared database abstraction) DRY within module, not outside May use an internal container Does not touch other module’s data directly Shared Services Connects the modules and serves as anti- corruption layer Only services you really think should not be duplicated Designed according to Dependency Inversion Principle Covers front-end and back- end is the scope of unit & integration tests Align folder structure with boundaries Defines the contracts for the host as well as any services it exposes Aligned with Bounded Context.
  5. 5. What is a microservice? From https://martinfowler.com/articles/microservices.html
  6. 6. Very scalable, very reliable Very difficult to debug distributed problems Unreliable network requires resiliency techniques Requires message bus/broker/ gateway End-to-end testing very unpractical Very mature DevOps culture is a prerequisite Advanced monitoring and visualization is a must High operational maintenance Can be deployed/upgraded independently Explosion of network activity.
  7. 7. The First Law of Distributed Computing
  8. 8. Monolith Functional Module Functional Module Functional Module HTTP API HTTP API HTTP API Released as NuGet packageNo network I/O at all Own database instance, / shared with monolith / or shared storage service Owns the schema Separate repos with distinct owners Runs in-process, thus easier debugging New version requires redeployment of the monolith. Loose coupling through HTTP Less scalability options HTTP is not the fastest serialization format Designed for minimum dependencies
  9. 9. Client IIS Console WinSvc Unit Test HTTP request HTTP response Middleware environment handler next Middleware IDictionary<string, object> Func<IDictionary<string, object>, Task> Task Func< Func<IDictionary<string, object>, Task>, Func<IDictionary<string, object>, Task> > Func<IDictionary<string, object>, Task>
  10. 10. Client IIS Console WinSvc Unit Test HTTP request HTTP response Middleware context handler next Middleware HttpContext Task Func< HttpContext, Func<Task>, Task > Func<HttpContext, Task> Func<HttpContext, Task>
  11. 11. Monolith Functional Module HTTP API Module Registry Contract documented through OpenAPI Webhook Functional Module Functional Module Provides HTTP abstraction to connect to module Event payload exposed as OpenAPI Subscribers register HTTP abstraction Consumers use liberal JSON interpretation Subcribers can request resending history. Payload includes unique ID to help idempotency Each request requires correlation ID that flows through all APIs, webhooks and module
  12. 12. Dennis Doomen | @ddoomen | The Continuous Improver Application Domain NoSQL / RDBMS OR/M / DAL Web UI, HTTP API, etc Lucene Index Document Projector Web UI, HTTP API, etc Web UI, HTTP API, etc Domain Commands Events Event StoreProjections Projectors Uses Event Sourcing Uses traditional CRUD architecture Indexing-based architecture Subcribe to webhooks Coarse- grained HTTP requests. Bus Subscribe Publish coarse- grained event
  13. 13. Domain is persistency ignorant For modules that provide a (partial) UI or management screen Aggregates Command Handlers Commands Events Upconverters Value Objects Query Handlers Schema Migrations Projectors Data Importers Query API Projections HTML CSS JS API Controllers Projectors Projections Registrations Webhook API Command API Event Storage Domain Services Payload Definitions Can be tested independently Based on Domain Driven Design Outer layers depend on inner layers Master data
  14. 14. Application Command Service Correct Customer Email Handler Customer Unit of Work Projector Data Access Layer Read Database Write Database CorrectCustomerEmailCommand HTTP API / In-process invocation Get<Customer>(identity) Correct Email Event Store Load(events) Apply Get changes CustomerEmailCorrectedEvent Submit changes History Dennis Doomen | @ddoomen | The Continuous Improver NES Aggregates.NET SimpleDomain NStore EventStore NEventStore SQLStreamStore NStore NES Marten Handelier Brighter MediatR (*) Projac LiquidProjections (*)
  15. 15. Monolith Functional Module HTTP API Event Store Events Payload Projector Subscription Request Payload ‘event store’ Have their own checkpoints Projects fine-grained events into payload projections Projector Projector Projector Webhook Projectors Separate instance per subscription Subscriber Callback API ‘Project’ payload ‘events’ into HTTP calls Projected Data Uses the incoming payloads as an ‘event store’. Payload SQL DB Schema changes using code or using NoSQL Subscription Request
  16. 16. - Module owns ‘schema’ - Supports upgrading and downgrading - NoSQL preferred, but RDBMS can work too.
  17. 17. Event Store Projector Application Version 2 Owns schema (e.g. FluentMigrator) Version 1 X Involves down-time until projections are rebuild Dennis Doomen | @ddoomen | The Continuous Improver
  18. 18. Event Store Projector Application Application Network Load Balancer Event Store Version 1 Version 2 events Projection ProjectorProjection bring off-line Returns HTTP 503 (Service Unavailable) Dennis Doomen | @ddoomen | The Continuous Improver Returns HTTP 503 (Service Unavailable)
  19. 19. Dennis Doomen | @ddoomen | The Continuous Improver
  20. 20. var mapBuilder = new EventMapBuilder<MyContext, DocumentProjection, string>(); mapBuilder .Map<WarrantAssignedEvent>() .AsCreateOf(anEvent => anEvent.Number) .Using((document, anEvent) => { document.Type = "Warrant"; document.Kind = anEvent.Kind; document.Country = anEvent.Country; document.State = anEvent.InitialState; }); mapBuilder .Map<StateTransitionedEvent>() .When(anEvent => anEvent.State != "Closed") .AsUpdateOf(anEvent => anEvent.DocumentNumber) .Using((document, anEvent) => document.State = anEvent.State); Dennis Doomen | @ddoomen | The Continuous Improver
  21. 21. var stats = new ProjectionStats(() => DateTime.UtcNow); stats.StoreProperty("CountByDocument", "some setting key", "some value"); stats.LogEvent("CountByDocument", "some significant thing that happened"); stats.TrackProgress("CountByDocument", currentCheckpoint); float? transactionsPerSecond = stats.GetSpeed("CountByDocument"); TimeSpan? eta = stats.GetTimeToReach("CountByDocument", targetCheckpoint); public void Configure(IAppBuilder builder) { builder.UseStatistics(stats); } Dennis Doomen | @ddoomen | The Continuous Improver
  22. 22. IAppBuilder builder = …. builder.UseStatistics(stats); GET http://localhost/projectionStats/CountByDocument { "projectorId": "CountByDocument", "lastCheckpoint": 1000, "lastCheckpointUpdatedUtc": "2018-05-10T10:39:00Z", "properties": [{ "key": "some setting key", "value": "some value", "lastUpdatedUtc": "2018-05-10T10:39:00Z" }], "eventsUrl": "http://localhost/projectionStats/CountByDocument/events" } Dennis Doomen | @ddoomen | The Continuous Improver
  23. 23. Service Host Monolith Functional Module Functional Module Functional Module HTTP API HTTP API HTTP API Apply routing conventions Versioning using Accept Headers, Response Headers or Routes Optimize body using AVRO or Protobuf Cross-service tracing using OpenTracing Replace HTTP with gRPC Common contracts for diagnostics, monitoring HTTP Proxy Child process of monolith (e.g. LitteForker) Hides the externalized state of the module (e.g. ProxyKit)
  24. 24. 1 Build a (container-based) cloud capability. 2 Deploy monolith to the cloud. 3 Automate your delivery pipeline. 4 Full end-to-end responsibility 5 Evaluate status-quo on micro-services (products, platforms, etc) 6 Slide-and-dice microservices one-by- one 7 Break-up delivery teams and code.
  25. 25. Dennis Doomen @ddoomen Dennis Doomen | @ddoomen | The Continuous Improver
  26. 26. • Liquid Projections https://www.liquidprojections.net • Bliki: MonolithFirst https://dzone.com/articles/martin-fowler%E2%80%94bliki • In-process gRPC https://github.com/grpc/grpc-go/issues/906 • Ingredients for well-designed OWIN components https://www.continuousimprover.com/search/label/OWIN%20 Recipes • The good, the bad and the ugly of Event Sourcing https://www.continuousimprover.com/search/label/event%20s ourcing
  27. 27. • Aggregates.NET https://github.com/volak/Aggregates.NET • EventFlow https://github.com/eventflow/EventFlow • OWIN in ASP.NET Core https://docs.microsoft.com/en- us/aspnet/core/fundamentals/owin?view=aspnetcore-2.1 • Nested ASP.NET Core applications http://dhickey.ie/2018/06/09/aspnet-core-nested-apps/ • LittleForker https://github.com/damianh/LittleForker • ProxyKit https://github.com/ProxyKit/ProxyKit
  28. 28. Domain Event Store Events App RDBMS Events Projection EventsProjection Projection Projector Optimized for specific queries Separate projections database NoSQL Projector Projection-specific storage Projector HTML Raw SQL or Dapper Run asynchronously Great for sharding Dennis Doomen | @ddoomen | The Continuous Improver
  29. 29. Events Transaction 6 Transaction 5 Transaction 4 Transaction 3 Transaction 2 Transaction 1 Temporal Projector Read Store Time Projected until this point Immutable, thus auditable and SOX compliance Dennis Doomen | @ddoomen | The Continuous Improver
  30. 30. Domain Event Store Events App Events Projection EventsProjection Projection Projector Application projections RDBMS Reporting Projector Traditional reporting model Asynchronous OLAP Dennis Doomen | @ddoomen | The Continuous Improver
  31. 31. Dennis (v4) Persisted State Changes Dennis (v3) Role Granted PasswordChangedEvent Dennis (v4) Dennis (v3) User Created Role Granted Phone Number Added Password Changed Dennis (v5) Role Revoked Dennis (v6) Role Granted Time Dennis (v7) Password Changed Dennis Doomen | @ddoomen | The Continuous Improver
  32. 32. Events 1 2 3 6297 6298 6298 Projections Documents Groups Users Objects Folders Microservice V2 Changes Queries Events 1 2 3 6299 6300 6301 Projections Documents Groups Users Objects Folders Microservice V1 ChangesQueries Migration Process Dennis Doomen | @ddoomen | The Continuous Improver
  33. 33. Events Aggregate Root Entity Entity Value Object Aggregate Root Aggregate Root Aggregate Root Entity Value Object Value Object Dennis Doomen | @ddoomen | The Continuous Improver
  34. 34. Designing your domain based on ownership
  35. 35. Relying on eventual consistent projections
  36. 36. Bad choice in functional keys (e.g. username vs ID)
  37. 37. Running business logic after an domain event is raised
  38. 38. Domain-specific value types in events
  39. 39. Ask questions about consistency Understand the real world
  40. 40. Driven by invariants, not composition Only consistent within aggregate Small aggregates Reference by identity
  41. 41. • Column constraints (e.g. data truncation) • Changes in data invariants (null vs non- null in event versions) • Unexpected projection dependencies • Partial replays.
  42. 42. • Causing duplicate child records • Causing large event streams • Incorrect caching strategy (e.g. on lookups) • Identity case-sensitivity • Incomplete SQL-backed event store reads.
  43. 43. • Side by side rebuilding • Functional archivability and archiving • Projection tracking and ETAs • Prioritization • Partitioning.
  44. 44. • After bugs, schema changes, etc • Manual or automatic (e.g. hashes)
  45. 45. Microservice Bunch of classes that should be used directly X No inheritance Uses composition over inheritance internally Convenience classes that don’t hide the magic Library Abstract Package Interfaces Delegates DTOs Only depend on abstract packages… Stable Package …or depend on more stable packages Auxiliary Package Classes that are not used together do not belong togetherOptional Dependency Dependency Package Consumers should not be faced with optional dependencies
  46. 46. public interface IEventStore { IDisposable Subscribe( long? lastCheckpoint, Func<Transaction[]>, Task> handler, string subscriptionId); } public delegate IDisposable CreateSubscription( long? lastCheckpoint, Func<Transaction[]>, Task> handler, string subscriptionId);
  47. 47. e.g. LibLog, TinyIoc, FluidCaching
  48. 48. • The network is reliable • Latency is zero • Bandwidth is infinite • The network is secure • Topology doesn't change • There is one administrator • Transport cost is zero • The network is homogeneous. https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

×