The document discusses software architecture principles for cloud infrastructure, including microservices, distributed computing fallacies, designing for failure, and new design patterns like cache-aside, circuit breaker, and event sourcing. It also covers topics like autoscaling, asynchronous messaging, reactive streams, configuration management, and challenges like software erosion and failures cascading in distributed systems. The overall message is that building distributed systems on cloud infrastructure requires adopting new architectural patterns to deal with failures and improve scalability, performance and resilience.

1. Software ArchitectureSoftware Architecture
for Cloud Infrastructurefor Cloud Infrastructure

2. Tapio RautonenTapio Rautonen
@trautonen
github.com/trautonen
fi.linkedin.com/in/trautonen
software architect
Enabling cloud superpowers in software development.

3. Cloud computing characteristicsCloud computing characteristics
On-demand self-service
Consumer can provision computing capabilities without requiring human interaction
Broad network access
Capabilities are available over the network and accessible by heterogeneous clients
Resource pooling
Provider's computing resources are pooled to serve multiple consumers dynamically
Rapid elasticity
Capabilities can be elastically provisioned and appear unlimited for the consumer
Measured service
Automatically controlled and optimized resources by metering capabilities

4. Software architecture principlesSoftware architecture principles
● Intentional architecture with emergent design
● High modularity
– high cohesion, loose coupling
– low algorithmic complexity
● Well described elements
– expressive and meaningful names and APIs
– clean code
● Passes all defined tests or acceptance criteria
● Lightweight documentation

5. Software architectureSoftware architecture  cloud computingcloud computing
MicroservicesMicroservices

6. Distributed computing fallaciesDistributed computing fallacies
1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous
Peter Deutsch / Sun Microsystems

7. Design for failureDesign for failure

8. New era of design patternsNew era of design patterns
● Cache-Aside
● Circuit Breaker
● Compensating Transaction
● Command and Query Responsibility Segregation (CQRS)
● Event Sourcing
● Queue-Based Load Leveling
● Sharding
● Throttling

9. Cache-Aside patternCache-Aside pattern
● Aggregated search combining multiple services
– requires additional search cache (Solr, ElasticSearch, ...)
● Improve performance of frequently
read data
● Local caching results
inconsistent state between
instances
● Consistency of data stores
and cache is really hard to
maintain
There are only two hard things
There are only two hard things
in Computer Science: cache
in Computer Science: cache
invalidation and naming things.
invalidation and naming things.
–– Phil Karlton
Phil Karlton

10. Circuit Breaker patternCircuit Breaker pattern
● Cloud infrastructure and distributed systems allow remote
services to fail in ways beyond imagination
– prevent failures to cascade
– allow system to operate in degraded mode
● Suitable for big microservices architecture
– creates routing complexity and overhead
– potential single point of failure  must be highly available
● Enables central logging and metrics
– dashboards and central state awareness

11. Circuit Breaker patternCircuit Breaker pattern
● During normal operation, breaker is in Closed state
– failures will eventually trip breaker
● While in Open state
– calls will fail fast and after some period attempts to reset
● In Half-Open state
– on successful call resets breaker, otherwise trips breaker
http://doc.akka.io/docs/akka/snapshot/common/circuitbreaker.html

12. Netflix Hystrix dashboardNetflix Hystrix dashboard

13. Compensating Transaction patternCompensating Transaction pattern
● Irrecoverable failures in distributed systems are hard
– eventual consistency, rollbacks are impossible
● Distributed transactions (XA)
– difficult and complex to implement
– still not bulletproof
– not usable for generic REST services
● Undo the effects of the original operation
– defines an eventually consistent steps for a reverse operation
– compensation logic may be difficult to generate
– operations should be idemponent to prevent further catastrophe

14. CQRS patternCQRS pattern
● Command and Query Responsibility Segregation
– segregates read and write operations with separate interfaces
– allows to maximize performance, scalability and security
● Introduces flexibility at the cost of complexity
– traditionally same DTO is used for read and write operations
– different data model for read (query) and write (command)
– supports different read and write data stores
– not suitable for simple business rules where CRUD is sufficient
● Often used together with event sourcing pattern

15. Event Sourcing patternEvent Sourcing pattern
● Append only store of events that describe actions for data
– simplifies tasks in complex domains by avoiding synchronization
– improves performance, scalability and consistency for
transactional data
– can serve multiple different materialized views
● Maintains full audit trail and history
– enables compensation actions
– supports play back at any point in time
● Events are simple
– the operation logic they describe might not be
– updates and deletes must be implemented with compensation
– “at least once” publication requires idemponent consumers

16. Queue-Based Load Leveling patternQueue-Based Load Leveling pattern
● Buffer between task and service
– minimizes the impact of peaks of work load
– task flood may result unresponsive or failure of the service
● Task provider and service runs asynchronously
– queue decouples tasks from the service
– service can handle tasks at its own optimal pace
– requires a mechanism for responses if the task expects a reply
Service
Tasks
Message queue

17. Sharding patternSharding pattern
● Divide data store into multiple horizontal partitions
– improves scalability when handling large volumes of data
● Overcomes limitations of single server data store
– finite storage space
– computing resources for large number of concurrent users
– network bandwidth governed performance
– geographically limited storage for legal or performance reasons
● Strategy defines the sharding key and data distribution
– wrong sharding strategy results bad performance
– balancing shards is not trivial, rebalancing is expensive
– referential integrity and consistency is hard to maintain
● Configuring and managing big set of shards is a challenge

18. Throttling patternThrottling pattern
● Controls the consumption of resource used by a service
– allows the system to function and meet SLA on extreme load
● Throttle after soft limit of resource usage is exceeded
– reject requests for user that exceed the soft limits
– disable or degrade functionality of nonessential services
– queue-based load leveling with priority queues
● Throttling is an architectural decision
– impacts the entire design of the system
– must be detected and performed very quickly
– services should return specific error code for clients
– can be used as an interim measure while autoscaling

19. SaaS architecture methodologySaaS architecture methodology
● Declarative formats for setup and runtime automation
● Clean contract with infrastructure for maximum portability
● Cloud platform deployments, obviating the need for ops
● Tooling, architecture and dev practices support scaling
Modern software is delivered from the cloud
to heterogeneous clients on-demand

20. The Twelve-Factor AppThe Twelve-Factor App
I. Codebase
one codebase tracked in revision control, many deploys
II. Dependencies
explicitly declare and isolate dependencies
III. Config
store config in the environment
IV. Backing Services
treat backing services as attached resources
V. Build, release, run
strictly separate build and run stages
VI. Processes
execute the app as one or more stateless processes
http://12factor.net/

21. The Twelve-Factor AppThe Twelve-Factor App
VII. Port binding
export services via port binding
VIII.Concurrency
scale out via the process model
IX. Disposability
maximize robustness with fast startup and graceful shutdown
X. Dev/prod parity
keep development, staging, and production as similar as possible
XI. Logs
treat logs as event streams
XII. Admin processes
run admin/management tasks as one-off processes
http://12factor.net/

22. Break the monolith in piecesBreak the monolith in pieces
● Monoliths come with a burden
– cognitive overload for developers
– scaling and continuous deployment becomes difficult
– long-term commitment to technology stack
– allows taking shortcuts for architectural design
If you can't build a monolith, what makes
you think microservices are the answer?

23. AWS reference architectureAWS reference architecture

24. Service discoveryService discovery
● Services need to know about each other
– inexistence of centralized service bus
– smart endpoints and client side load balancing
● Service registry is the new single point of failure?
– value availability over consistency
● Provides a limited set of well defined features
– services notify each other of their availability and status
– cleaning of stale services
– easy integration with standard protocols like HTTP or DNS
– notifications on services starting and stopping

25. Ephemeral runtime environmentsEphemeral runtime environments
● Short lifetime of an application runtime environment
– scaling, testing, materializing ideas
– requires highly automatized infrastructure
● Nothing can be stored in the runtime environment
– logs, file uploads, database storage files, configuration
● Results stateless services
– optimal for horizontal scaling
– integrates to State as a Service
● Must be repeatable and automatically provisioned

26. Metrics and loggingMetrics and logging
● Ephemeral and dynamic systems
– requires central awareness of state
– audit logging of changes in the system
● Gain understanding how the services are used
– plan for future requirements
– gather scaling metrics
– bill customers for usage (pay-per-use)
– detect faulty behavior
● Balance between value provided and cost of collecting
– robustness of the metering system impacts on profitability
– collect end-to-end scenarios rather than operational factors

27. AutoscalingAutoscaling
● Adapting to changing workloads
– optimize capacity and operational cost
– increase failure resilience
● Requires key performance metrics capturing
– response times, queue sizes, CPU and memory utilization
● Decision logic based on scaling metrics
– when to scale up and down
– prevent scaling oscillation
● Application must be designed for scaling
– stateless, immutable, automatically provisioned

28. Asynchronous messagingAsynchronous messaging
● Key strategy for services to communicate and coordinate
– decouple consumer process from the implementing service
– enables scalability and improves resilience
● Basic messaging patterns
– sender posts a one-way message and receiver processes the
message at some point in time
– sender posts a request message and expects a response
message from the receiver
– sender posts a broadcast message which is copied and
delivered to multiple receivers
● Numerous implementation concerns
– message ordering, grouping, repeating, poisoning, expiration,
idempotency and scheduling

29. Reactive streamsReactive streams
● Originates from The Reactive Manifesto
– Responsive system responds in a timely manner
– Resilient system stays responsive in the face of failure
– Elastic system stays response under varying workload
– Message Driven system relies on asynchronous messaging
● Initiative to provide standard for asynchronous stream
processing with non-blocking back pressure
– minimal set of interfaces and methods to achieve the goal
● Collaboration of people from high profile companies
– Typesafe, Oracle, Pivotal, Netflix, Red Hat, Applied Duality, ...
● Akka Streams, Reactor Composable, RxJava, Ratpack
https://www.coursera.org/course/reactive

30. Data consistencyData consistency
● All instances of application see the exact same data
– strong consistency
● Application instance might see data of operation in flight
– eventual consistency
● Distributed data stores are subjected to CAP theorem
– consistency, availability, partition tolerance
– only two of the features can be implemented
● Recovering from failures of eventually consistent data
– retry with idemponent commands
– compensating logic

31. Configuration managementConfiguration management
● Externalize configuration out
of runtime environment
– repeatable, versioned
● Local configuration pitfalls
– limits to single application
– hard for multiple instances
● Runtime reconfiguration
– application can be reconfigured without redeployment or restart
– minimize downtime, enable feature flags, help debugging
– thread safety and performance is a concern
– prepare for rollbacks and unavailability of configuration store

32. Software erosionSoftware erosion
● Slow deterioration of software leading to faulty behavior
● Fighting erosion is more expensive than usually admitted
● Erosion-resistance comes from separation of concerns
– application – infrastructure
● Clear contract of services provided by infrastructure
– change in infrastructure does not break the contract
– application can change within its respected realm
● Solutions against erosion
– Platform as a Service
– container virtualization

33. Cloud architecture pitfallsCloud architecture pitfalls
● Failures do cascade
– even without a single point of failure
● Multi-service search is hard to get right
– cache-aside issues
● Never rely on unreliable message delivery
– use asynchronous persistent message stores
● Monolith has one big problem
– microservices will generate a lot of small (and big) problems
● Do not ignore the platform's managed resources
– but evaluate the lock-in risk

34. Reach for the skiesReach for the skies
● Distributed systems are hard to build
– no silver bullet exists (sorry to disappoint again)
● Cloud infrastructure drives towards microservices
– start with a monolith, expand to microservices
– learn new design patterns during the journey
– automated system requires less ops and offers more resilience
● Do you think Netflix did it right the first time?
– learn from failure
– design for failure
● Cloud native applications are the future

35. Thank youThank you