This document discusses patterns and principles for distributed application design. It covers basic concepts like client-server and multitier architectures. It discusses scaling issues like load balancing and sharing of resources. It then describes evolving to a more service-oriented approach with independently deployable services behind an API gateway and using service discovery. It covers patterns for communication, data consistency using eventual consistency with event sourcing and CQRS, and ensuring high availability. The key aspects are to think about security, high availability, communication patterns, and leveraging data-oriented microservices with eventual consistency between command and query models.

1. Distributed Application Design
Patterns & Principles

2. Basic Concepts
Problem Statement

3. Application
• A set of features delivered to end users.

4. Client-Server
• Client – the application used by end users.
• Server – the backend application used by client apps.

5. Multitier
• A client-server app with physically separated layers.
• Presentation;
• Application Processing;
• Data Management;

6. Excessive client connections
• How do we scale if there are too many client connections?

7. Load Balancer
• Scale to multiple instances behind a load balancer.
• Potential bottlenecks… database?

8. Scalability
• What is shared?
• Clients share Server apps;
• Server apps share Database;

9. Scalability
• What is shared?
• Clients share Server apps;
• Server apps share Database;
• Shared resources:
• CPU, Memory, Disks...
• What else?

10. Scalability
• What is shared?
• Clients share Server apps;
• Server apps share Database;
• Shared resources:
• CPU, Memory, Disks...
• What else?
• Pareto principle:
• 80/20 rule.

11. Evolution
Service-Oriented Approach
(not Architecture, not SOA, but SO*)

12. Service-Oriented
• Services are deployed separately.

13. Service-Oriented
• Services are scaled independently.

14. Cascading Calls
• Services may reuse other services.

15. Load Balancing
• Do we need a Load Balancer per Service?

16. Service Discovery
• Service Registry:
• Services registers at startup and provide heartbeats.
• Registry knows location of each services (all instances).
• Services access services directly, Registry is an address-book, not proxy.

17. Load Balancing with Service Discovery
• Server-side Load Balancing:
• Discovery Server returns a single instance (e.g. round-robin).
• Client-side Load Balancing:
• Client fetches list of all instances and decides which to use.

18. Discovery Cluster
• Single point of failure:
• Discovery Server (service registry) may fail.
• Discovery Server Cluster is required.

19. Dynamic Scalability
• Deploy more instances to scale on the fly.

20. API Gateway
• Proxy API or Expose specific API (a service as API Gateway).
• Or proxy always and create new services for specific API.

21. Patterns so far…
• Service Discovery
• Client-side
• Server-side
• API Gateway
• Proxy
• Programmed
• Load Balancer
• Client-side
• Server-side

22. High Availability

23. High Availability

24. High Availability

25. High Availability

26. High Availability

27. High Availability

28. High Availability

29. • Regions
High Availability

30. • Availability Zones
High Availability

31. • All services available.
High Availability

32. • All instances of a service are not available in local zone.
High Availability

33. • All instances of a service are not available in local region.
High Availability

34. • All services recovered after fail.
High Availability

35. Architectural Styles

36. Layers
• Services are organized into layers, each layer performing a specific
role within the application.

37. Microservices
• Services form a single application.
• Independently deployed, horizontally scaled and dynamically coordinated;

38. SOA
• Applications are provided to the other components as services.
• Support independence of vendors, products and technologies.

39. Security Concerns

40. Authentication
• Authentication is the process of confirming the identity.
• Is requestor really represents the identity claimed?

41. Authorization
• Authorization is the function of specifying access rights to resources.
• Does requestor have all necessary privileges?

42. Communication
• Data could be stolen while being transmitted.
• Is communication secure enough?

43. Implementation Details
• Roles available:
• User, Client;
• Auth Service;
• Service Provider;
• Common concerns:
• Latency;
• Bottlenecks;
• Statelessness;
• Single Sign-On;
• Fine-grained authorization;
• Exposure of user credentials;

44. Capability-Based
• A capability (key) is a communicable, unforgeable token of authority.
• It references an object along with an associated set of access rights.
• It is used to access another object.

45. Protection Rings
• System is secured using protection rings.
• Diametrically opposite to capability-based security.

46. Communication

47. Direct
• Services communicate with other services directly:
• Coordinated by a discovery server for dynamic scalability.
• Client-side load balancing option available.
• Requires strict contract.
• Remote services should be highly available.
• May cause data loss in case if requests fail.

48. Decoupled
• Services communicate though a middleware:
• Coordinated by the middleware.
• Scaled using such concepts as Partitioning & Consumer Groups.
• Data-oriented, almost contract-free.
• Middleware should be highly available.
• No data loss, middleware can keep it.

49. Synchronous
• Block and wait for answer upon request.
• Standard model to fetch data.
• Blocking API.

50. Asynchronous
• Fire-and-forget model.
• Best suited for action triggers.
• Non-blocking API.

51. Communication
Direct Decoupled
Synchronous
Asynchronous

52. Data Microservices
The Concept

53. Data Microservices
• Basic properties:
• A lightweight data-oriented service;
• Has an input and/or output;
• May play role of sink, source or processor;
Source O SinkIProcessorI O

54. Data Microservices
• Advanced properties:
• Almost contract-free (no methods, just data);
• Decoupled (know nothing about others);
• Reusable (anytime, anywhere);
• Used in composition of pipelines (workflow);

55. Data Microservices: Scalability
• Consumer Groups.
• Scale the architecture at consumer side when there is a demand for more
computational power (high volume of incoming messages, slow consumer).
Publisher
Group: Grey
Consumer 1
Group: Yellow
Consumer 1 Consumer 2Consumer 2

56. Data Microservices: Scalability
• Partitioning.
• Scale the architecture at producer side when there is a demand for more
parallelism at middleware.
Publisher 2
Group: Grey
Consumer 1
Group: Yellow
Consumer 1 Consumer 2Consumer 2
Publisher 1 Publisher 3
1 2 3

57. Data Consistency

58. Transactional Consistency
• Using a transaction manager.

59. Transactional Consistency
• Using a transaction manager.

60. Transactional Consistency
• Using a transaction manager.

61. Transactional Consistency
• Using a transaction manager:
• We heavily depend on it;
• It delays the processing of our transactions (potential bottleneck);
• Complex to implement;
• More complex when we need to communicate with another system.

62. Transactional Consistency
• When dealing with transactions:
• Think about different approaches for different business processes;
• A transaction manager with two phase commit is not always required;
• Actor/event oriented approaches are available as well.
• Coffee shop example.

63. Eventual Consistency
• Event Sourcing:
• Save events of state changes instead of state itself.
• Reconstruct an entity’s state by replaying events.
• Saving an event is a single operation, it is inherently atomic.
Create Account Edit User Name Delete Account

64. Eventual Consistency
• Event Store:
• Events are immutable, store full history, provide a detailed audit trail.
• Events are published to subsystems notifying about the changes to the state.
• Rewind to the state of the system at any previous point in time.
Event Store
Group: Grey
Consumer 1
Group: Yellow
Consumer 1 Consumer 2Consumer 2

65. Eventual Consistency
• Retrieval of state is time consuming.
• Have to replay events to rebuild the state.
• Solution is to maintain two separate models:
• write model — the event store.
• read model — the database.
• A different handler is responsible for updating the read model, once an event
is persisted in the event store (write model).
Write
Model
Read
Model
Events

66. Eventual Consistency
• CQRS:
• every method should either be a command that performs an action,
• or a query that returns data to the caller.
• Return a value only if there are no side effects.
• If you have a return value you cannot mutate state.
• If you mutate state your return type must be void.
Command Query

67. Eventual Consistency
• Event Sourcing + CQRS
Command
Service
Query
Service
Event
Store
Data
Store
Event
Processor

68. Eventual Consistency
• Event Sourcing + CQRS
• Consistency:
• Write model is consistent, events are ordered, transactions are simple.
• Most systems can be eventually consistent on the Query side.
Command
Service
Query
Service
Event
Store
Data
Store
Event
Processor

69. Eventual Consistency
• Event Sourcing + CQRS
• Consistency:
• Write model is consistent, events are ordered, transactions are simple.
• Most systems can be eventually consistent on the Query side.
• Data Storage:
• Normalized data on the Command side (e.g. a relational structure).
• Denormalized data on the Query side (e.g. to minimize the number of joins).
Command
Service
Query
Service
Event
Store
Data
Store
Event
Processor

70. Eventual Consistency
• Event Sourcing + CQRS
• Consistency:
• Write model is consistent, events are ordered, transactions are simple.
• Most systems can be eventually consistent on the Query side.
• Data Storage:
• Normalized data on the Command side (e.g. a relational structure).
• Denormalized data on the Query side (e.g. to minimize the number of joins).
• Scalability:
• Scale Command and Query sides separately.
Command
Service
Query
Service
Event
Store
Data
Store
Event
Processor

71. Summary

72. Summary
API Gateway
Discovery
Servers
Command
Service
Query
Service
Event
Store
Data
Store
Data
Store
Command
Service
Command
Service
Query
Service
Query
Service

73. Summary
• Think about:
• Security
• High Availability
• Communication
API Gateway
Discovery
Servers
Command
Service
Query
Service
Event
Store
Data
Store
Data
Store
Command
Service
Command
Service
Query
Service
Query
Service

74. Live Coding
Using Spring Cloud

75. Live Coding
• Phase 1: Application.
• Endpoint to Hello a user by name.
• Endpoint to Hello the world.
Hello
App
Mood
Ext. Service

76. Live Coding
• Phase 2: Service-Oriented.
• API Gateway.
• Service Discovery.
• Load Balancing.
Hello
API Gateway
Mood
Ext. Service
Hello World
Service
Hello
Service
Discovery
Mood
API Service

77. Live Coding
• Phase 3: Eventually Consistent.
• Event Sourcing.
• CQRS.
• Post-Redirect-Get.
Hello
API Gateway
Mood
Ext. Service
Hello World
Service
Hello Query
Service
Discovery
Mood
API Service
Hello
Command
Service
Hello
Decorator
Mood
Decorator
Data
Store

78. Live Coding

79. Thank You!
Patterns of Distributed Application Design