The document discusses the evolution of web application architectures from single server deployments to modern cloud-native microservice architectures. It covers how early web apps faced challenges as demand grew that led to adopting caching, load balancing, and the cloud. It then discusses how further scaling issues were addressed through containers, microservices, and patterns like API gateways and circuit breakers. Key lessons are around adopting cloud-native patterns to improve performance, scalability, resilience and developer productivity for applications.

1. Finding Patterns
in the
Clouds
Steve “ardalis” Smith
@ardalis | steve@ardalis.com
ardalis.com | weeklydevtips.com
Design Patterns for Cloud-
Native Applications

2. Please Rate in the App
• AttendeeHub
@ardalis | Finding Patterns in the Clouds

3. More Resources
• Podcast
WeeklyDevTips.com
• Group Mentoring Program
DevBetter.com
• Free Microsoft eBooks
ardalis.com/architecture-ebook
ardalis.com/cloud-native-book
@ardalis | Finding Patterns in the Clouds

4. Let’s take a trip back to the
beginning of today’s web…
@ardalis | Finding Patterns in the Clouds

6. (not that far)

7. A Simpler Time
“Just” ~20 years ago…
Compaq AlphaServer DS20 circa 1999
@ardalis | Finding Patterns in the Clouds

8. A Simpler Time
“Just” 20 years ago…

9. One Web Server To Rule Them All
Client Machine Server
Request
Response
App
Data
(and one Webmaster to run it all – the original Full Stack Developer™)
@ardalis | Finding Patterns in the Clouds
NCSA Mosaic

10. One Web Server To Rule Them All
Client Machine Server
Request
Response

11. Report Card: One Server To Rule Them All
@ardalis | Finding Patterns in the Clouds
Availability
Data Management
Design and Implementation
Messaging Security
Management and Monitoring
Resiliency
Performance and Scalability

12. Web App
Considerations and Challenges
@ardalis | Finding Patterns in the Clouds

13. Cloud-Hosted Web App
Considerations and Challenges
@ardalis | Finding Patterns in the Clouds

14. Availability
How often is the system or service up?
Often expressed as a percentage.
99.99% uptime = 1 minute of downtime per week
99.999% uptime = 26 seconds of downtime per month
@ardalis | Finding Patterns in the Clouds

15. Data
Management
Many more options than in traditional-hosted
single-database apps
Distributed data
Consistency
Synchronization
@ardalis | Finding Patterns in the Clouds

16. Design and
Implementation
Consistency is key
Consider factors like
• Maintenance ease
• Administration
• Development
• Diagnostics
• Cost
@ardalis | Finding Patterns in the Clouds

17. Messaging
How do subsystems communicate?
Direct, synchronous calls?
Asynchronous messaging?
Each option presents challenges.
@ardalis | Finding Patterns in the Clouds

18. Management
and Monitoring
No direct server access to PaaS resources
means other tools are critical.
Cloud resources are more like cattle herds
than pets.
@ardalis | Finding Patterns in the Clouds

19. Performance
and Scalability
How responsive is the system to requests?
How does this responsiveness change with
increased load?
Scaling up
Scaling out
@ardalis | Finding Patterns in the Clouds

20. Resiliency
Can the system gracefully (and automatically)
recover from errors or failures?
Detect failures and replace resources
automatically
@ardalis | Finding Patterns in the Clouds

21. Security
Protect from attacks
Guard sensitive data
Restrict access to approved users
@ardalis | Finding Patterns in the Clouds

22. One Machine To Rule Them All
Client Machine Server
Request
Response
App
Data
@ardalis | Finding Patterns in the Clouds

23. Report Card: One Server To Rule Them All
🙁
🙂
🙂
🙂
🙂
🙂 but also 🙁
😐
😐
@ardalis | Finding Patterns in the Clouds

24. Demand Grows

25. 🙁 Vertical Scaling is
Maxed Out
🙁 Performance is
suffering at times
Remedy:
Move Database to a
separate server
🙂 Performance
improves
Current Assessment:
@ardalis | Finding Patterns in the Clouds

26. 1 Web, 1 DB Server
Client Machine Web Server
Request
Response
App Data
DB Server
@ardalis | Finding Patterns in the Clouds

27. Parts of the
App are SLOW

28. 🙁 Vertical Scaling is
Maxed Out (both
servers) – or at least
there’s no budget for
more right now
🙂 Data has been
optimized with indexes,
etc. No more gains to be
had here.
🙁 Some queries just
hammer the database,
take time, and impact
other queries.
Current Assessment:
@ardalis | Finding Patterns in the Clouds

29. Cache Aside Pattern
@ardalis | Finding Patterns in the Clouds

30. Read-Through Strategy
@ardalis | Finding Patterns in the Clouds

31. Write-Through Strategy
Data
1. Update the data store 2. Invalidate (or update) its
cache entry
$123
@ardalis | Finding Patterns in the Clouds
Another option is to use a very short cache duration, something I refer to as micro-caching

32. Add Simple Memory Caching
Client Machine Web Server
Request
Response
App Data
DB Server
Cache
@ardalis | Finding Patterns in the Clouds

33. There are only 2 hard things in Computer Science
0. Cache Invalidation
1. Naming Things
2. Off-by-One Errors
@ardalis | Finding Patterns in the Clouds

34. Speaking of Naming Things…
• Use a standard way to generate a cache keys for given scenarios
• Avoid hard-coding keys, especially as local method literals
• Many caching patterns require access to keys from different parts of
your applications (including read vs. write operations)
@ardalis | Finding Patterns in the Clouds

35. 🙂 Performance is
usually (much) better
🙁 Some users still
complain of delays due
to cache misses
🙁 Keeping Cache
up to date is a new
challenge
🙁 Customers may
now see stale data
New Problems…
🙁 New monitoring
required for cache
🙁 New tools to clear
or update cache
required
Hmm, that’s a lot of 🙁

36. Demand Grows

37. Time to Scale Out!

38. Simple Web Farm
Client Machine
Web Server
Request
Response
App
Data
DB Server
Cache
Load Balancer
Web Server
App
Cache
@ardalis | Finding Patterns in the Clouds

39. 🙂Performance and
Scalability improved
🙁 Some users still
complain due to
cache misses
😧 Keeping Multiple
Caches up to date is
a big challenge (in
this model)
New Behavior…
🙁New monitoring
and tools required
for load balancer
🙁 More servers to
manage
🙂 Web servers can
be updated without
taking down the
whole system
@ardalis | Finding Patterns in the Clouds

40. Embrace the
Cloud!
@ardalis | Finding Patterns in the Clouds

41. The Cloud (xkcd.com/908)
@ardalis | Finding Patterns in the Clouds

42. Simple Cloud Architecture
Client Machine
Request
Response
Load Balancer
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
@ardalis | Finding Patterns in the Clouds

43. 🙂 Scalability
improved
🙁 Some users still
complain due to
cache misses
(consider priming
the cache)
🙂 Cache
synchronization easier
New Behavior…
🙂 No more servers
to manage
🙂 Monitoring tools
built-in to platform
🙂 Web instances
easily managed
without downtime
That ratio of 🙂 to 🙁 is a lot better…@ardalis | Finding Patterns in the Clouds

44. “Let’s build more of these apps”
@ardalis | Finding Patterns in the Clouds

45. More Apps
Client Machine
Request
Response
Load Balancer
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
@ardalis | Finding Patterns in the Clouds

46. More Apps
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
App A
App Service
Instances
App B
App Service
Instances
App C
Shared Resources

47. Everything’s Great! Except…
Vendor lock-in
Shared database hurts
Shared resources (e.g. data schema) limit app developer agility
We’ll address these in a moment but first…
@ardalis | Finding Patterns in the Clouds

48. “Don’t forget auth!”
@ardalis | Finding Patterns in the Clouds

49. Authentication and Identity
These apps require:
• Single Sign-on
• Security – protection from unauthorized use
• This was surely built into the apps before this point, but the pain
becomes apparent now
@ardalis | Finding Patterns in the Clouds

50. Simple Database Managed Identity
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
App A
App Service
Instances
App B
App Service
Instances
App C
Identity
Tables/Data
Username
Password
Login
Username
Password
Login
Username
Password
Login
YOU get a login screen, and YOU get a login screen

51. Federated Identity Pattern
@ardalis | Finding Patterns in the Clouds

53. Redis
Cache
Azure
SQL
App A
Redis
cache
Azure
SQL
App B
Redis
cache
Azure
SQL
App C
Identity
Microservice
Leveraging Containers and Federated Identity
Azure
SQL
Client Machine
1. Authenticate
2. Get Secure Token
3. Present Token
Note: Vendor lock-in mitigated by containers

54. “What about data sync?”
@ardalis | Finding Patterns in the Clouds

55. Redis
Cache
Azure
SQL
App A
Redis
cache
Azure
SQL
App B
Redis
cache
Azure
SQL
App C
Identity
Microservice
Implementing a Message/Event Bus
Azure
SQL
@ardalis | Finding Patterns in the Clouds

56. Moving from Apps to Microservices
• Apps are very coarse-grained to deploy, scale
• Common functionality duplicated between apps
• Stable parts of apps disrupted by deployment of unstable bits
• Decompose apps into small, independent, cohesive microservices
@ardalis | Finding Patterns in the Clouds

57. App A – An eCommerce Site
Redis
Cache
Azure
SQL
App A
Client apps
Mobile
app
Web
app
@ardalis | Finding Patterns in the Clouds

58. Microservice 2
Microservice 1
container
container
Web API
Web API
Microservice 3
container
Web API
Client apps
Microservices
Split into Microservices
Call each as appropriate from clients
Mobile
app
Web
app

59. Security Concerns
• Client apps may not need every
microservice feature
• Microservices may have multiple
clients; shouldn’t need to know
security rules of every one
• Should limit feature surface area
specific to client needs
@ardalis | Finding Patterns in the Clouds

60. API Gateway Pattern
@ardalis | Finding Patterns in the Clouds

61. Using a custom API Gateway Service
Microservice 2
Microservice 1
Client WebApp MVC
container
container
Web API
Web API
ASP.NET Core MVC
container
Microservice 3
container
Web API
Client SPA Web app
JavaScript
Client mobile app
API Gateway
ASP.NET Core
Web API
container
Back end
Traditional Web app
Browser
HTML
HTML
JSON
JSON

62. API Gateway with Azure API Management
Architecture
Client WebApp MVC
ASP.NET Core MVC
container
Client SPA Web app
JavaScript
Client mobile app
Developer portal
API Gateway
Publisher portal
Azure API Management
Microservice 2
Microservice 1
container
container
Web API
Web API
Microservice 3
container
Web API
Back end

63. Accessing Secure Files
@ardalis | Finding Patterns in the Clouds

64. New Problem – Secure File Access
• Apps control access to media files based on authorized user
• Simple approach of a dumb CDN doesn’t protect actual media URLs
from being accessed by anyone
• Current solution: Web App authenticates user, accesses the file, and
streams it to the end user
@ardalis | Finding Patterns in the Clouds

65. Secure Media File Access
Client Machine
Request
Response
Web App
Request
Response
File/BLOB Store
(not publicly
accessible)
@ardalis | Finding Patterns in the Clouds

66. Concerns
🙁 Load on web app
higher than necessary
🙁 Cost! 💰 May be
paying extra to move
files in/out of web
app
🙁 Greater chance
of downtime with
web app and file
store both required
to stream file
@ardalis | Finding Patterns in the Clouds

67. Valet Key
• Provide direct access to media
files using an access token
• Azure supports Shared Access
Signatures (SAS) for this purpose
• File transfers occur directly
between file store and client
@ardalis | Finding Patterns in the Clouds

68. Remote Resources
@ardalis | Finding Patterns in the Clouds

69. “I want fast, reliable, always-on services!”
@ardalis | Finding Patterns in the Clouds

70. Retry Pattern
First attempt failed
• Is it likely to be a transient problem? If not, give up.
• Immediately try again (maybe it was just a fluke)
• (wait)
• Try again
• (wait longer)
• Try again
• Give up.
@ardalis | Finding Patterns in the Clouds

71. Retries can overload downstream service
• Imagine typical load is 10 requests per second.
• With a typical “try 3 times then fail” strategy, when the service comes
up it’s immediately seeing 30 requests per second of load!
• Can result in longer time to respond and creation of more resources
than necessary (more hosting $$$)
• Don’t DOS (denial of service) yourself!
@ardalis | Finding Patterns in the Clouds

72. Circuit-Breaker Pattern
3 States
• Closed (working)
• Open (not working)
• Half-Open (throttled)

73. Circuit Breaker States
@ardalis | Finding Patterns in the Clouds
Closed
Open
When failCount > threshold, Open
Half-Open
After [timeout], go to Half-Open
Still not responding…
Reset failCount to zero and Close
Retry on failure; increment failCount

74. Consider a tool like Polly
@ardalis | Finding Patterns in the Clouds

75. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 1
• Microservices access reporting
data directly
• Introduces coupling
• Reduces microservice
independence
• Bypasses microservice logic
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x

76. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 2
• Reporting app hits
microservices directly
• Poor performance
• Data may be too large for
HTTP
• Difficult to perform complex
queries
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x

77. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 3
• Batch job updates reporting db
from microservice dbs
• Same coupling as approach 1.
Changes to microservice db
schemas break batch data job.
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x

78. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Solution
• Async event publication
• Encapsulation and
independence of microservices
is preserved
• Performance is usually
acceptable
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x

79. Key Takeaways
• Cloud architecture abstracts away servers
• Cache Aside pattern is great for performance improvements
• Containers offer improved deployment and scaling options with less
vendor lock-in
• Microservices offer finer-grained control over app functionality
• Federated Identity improves security and user experience
• API Gateways help secure collections of services
• Valet key provides cheaper, faster access to secure media
• Consider Retry but you may need a Circuit-Breaker
• Avoid the Reach In Reporting anti-pattern for your microservices
@ardalis | Finding Patterns in the Clouds

80. More Cloud Design Patterns
https://bit.ly/1T8q2w8
@ardalis | Finding Patterns in the Clouds

81. Thank You!
• Contact me!
twitter.com/ardalis
steve@ardalis.com
• Podcast
WeeklyDevTips.com
• Group Mentoring Program
DevBetter.com
• Free Microsoft eBooks
ardalis.com/architecture-ebook
ardalis.com/cloud-native-book