Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Testing large-scale, serverless
and asynchronous systems
Matt Fellows
@matthewfellows
...without end-to-end integrated tes...
2
Why Serverless?
https://aws.amazon.com/retail/data-analytics/
WHAT WE ARE KNOWN FOR
3
Microservices
Evolution Drivers for levelling up
● Increase time-to-value
● Increase utilisation /...
Trends
5
Data Pipelines
Trends > Data pipelines
6
Trends > Data pipelines
7 https://aws.amazon.com/blogs/big-data/automating-analytic-workflows-on-aws/
Trends > Data pipelines
8 https://www.slideshare.net/nateware/aws-gdc2015-gameanalyticsfinal
Trends > Data pipelines
9
Trends > Data pipelines > Before
Systems of Record
Integration / Middleware
APIs
Presentation
Synchronous,
coupled invoc...
10
Trends > Data pipelines > After
System of Record
Event
API
Local cache /
materialised view
API
3. Event processed into ...
Observations
Architecture > Microservices
● Democratisation of data
○ “Everyone gets a data”
● Hollywood Principle
● Pipel...
12
IT Ops. Automation
Trends > IT Ops Automation
Before...
● Ad-hoc
● Cron jobs running on instances
● Scripts > code
● Shared “Job” machines, with broad
security permissi...
14
Trends > IT Ops Automation
Decide
Alarm
Schedule
Event
subscription
Notify Pagerduty
Act
Isolate instance from
ASG
Ship...
New approach
● Scheduled Lambda to replace “cron” jobs
● Real-time cron - event-driven
● Function written and deployed as ...
16
The new uService
Trends > Microservices
17
Trends > Microservices > Before
Instances
Team 1
Team 2
Team 3
Team ...
18
API Gateway
Trends > Microservices > Before
Services
Services
Team 1: Orders Team 2: Identity
Services
Team ...
Team 3:...
19
Trends > Microservices > After > New Patterns
API
External
Events
API
Outside World
Microservice
Interface
API
Master D...
20
Provisioning
Amazon ML
Trends > Microservices > After > Example Team 1 Bounded Context
Place Order API
External
Events
...
21
Trends > Microservices > Evolution
Sizeofthing
Number of communicating things
Monolith
SOA
Serverless
Microservices
Fn
...
Observations
Architecture > Microservices
● Microservices still a thing
○ ..but are now just the interface of the
iceberg
...
Summary
Architecture > Microservices
● Serverless as a higher-order system
○ Composition
● IT Operations Automation
○ Bett...
24
Much Compose! So speed!
Architecture > Functions + Serverless (Cloud Native)Architecture > Functions + Serverless (Clou...
25
New challenges
Architecture > Functions + Serverless (Cloud Native)
26
If you can’t build a good monolith you
shouldn’t be doing microservices
Architecture > Microservices
...the µServices
ride
● Ability to write a modular monolith
● Strong grasp on your domain model
● Ability to design loosel...
28
If you can’t build good microservices
you shouldn’t be doing serverless
Architecture > Functions + Serverless (Cloud Na...
29
What practices do we need in this
new world?
Architecture > Functions + Serverless (Cloud Native)
Serverless Lounge
31
Demo: AWS Meetup Lounge
CloudFront
API Gateway
Authentication
You!
awslounge.onegeek.com.au
Twitter Fetch
Twitter feed
...
32
Demo: AWS Meetup Lounge
Demo
Challenges
Blasphemy!
34
Smaller functions = more things!
Challenges > Async
WHAT WE ARE KNOWN FOR
35
Options
● ...
Challenges
● Observability + Tracing
● Structured logging
● Shift responsibility in...
36
Vendor lock-in
Challenges > Vendor lock-in
37 https://twitter.com/alambert/status/958050456101642241
Challenges > Vendor lock-in
WHAT WE ARE KNOWN FOR
38
Options
● Multi-cloud = lowest
common denominator
● There is always a vendor
(even if it’s you!)
...
WHAT WE ARE KNOWN FOR
39
Options
● Multi-cloud = lowest
common denominator
● There is always a vendor
(even if it’s you!)
...
Lambda
Handler
Challenges > Vendor lock-in > Lambda Anti-pattern
{
"Records": [
{
"EventVersion": "1.0",
"EventSubscriptio...
MQ Handler
WebSocket
handler
MQ Adapter WS Adapter
Azure Function
Adapter
Google Cloud Fn
Adapter
Azure Function
Handler
G...
42
Challenges > Vendor lock-in > Ports and Adaptors > Sentiment Analysis Example
Twitter feed
Sentiment
Analysis
IoT Messa...
Challenges > Vendor lock-in > Ports and Adaptors > Lambda Port
Challenges > Vendor lock-in > Ports and Adaptors > SNS Adapter
Challenges > Vendor lock-in > Ports and Adaptors > Lambda Sentiment Service
Summary
Architecture > Microservices
● Separate protocol handling from
business logic
● Business logic shouldn’t change wi...
47
How do you test locally?
Challenges > Testing Locally
WHAT WE ARE KNOWN FOR
48
Options
Challenges > Testing Locally
● Stub services (e.g. Localstack)
● Multiple handlers
● Stub...
49
Challenges > Testing Locally
WHAT WE ARE KNOWN FOR
50
Options
● Feature arms race
● Trustworthy?
● Integration style = slower
● All cloud providers?
● ...
WHAT WE ARE KNOWN FOR
51
Writing good unit tests
● Follow test pyramid
● Stub out AWS APIs
● Test your business logic
sepa...
52
Challenges > Testing Locally > Unit Test > Twitter Fetch
Checkpoint
Twitter Fetch
Challenges > Testing Locally > Unit Test > Lambda Service
Look ma, no AWS
Code
Challenges > Testing Locally > Unit Test > Lambda Service
Look ma, no AWS
Code
55
Challenges > Testing Locally > Unit Test > Sentiment Analysis Example
Twitter feed
Sentiment
Analysis
Integration-style test
of handler
Challenges > Testing Locally > Unit Test > Lambda Handler
Summary
Architecture > Microservices
● Business logic should be tested
separately from port + adapter
● Majority of tests,...
58
Async - who is my consumer?
Challenges > Async
59
Async - delay in feedback
Challenges > Async
WHAT WE ARE KNOWN FOR
60
Options
● Local integration testing
● Cloud env. integration testing
● Contract testing
Challenge...
WHAT WE ARE KNOWN FOR
61
Options
● Can’t run complete local
environment
● Many moving parts
● Problems with integration
te...
WHAT WE ARE KNOWN FOR
62
Options
● Can’t run complete local
environment
● Many moving parts
● Problems with integration
te...
63
Contract tests
Challenges > Async
64
IoT Message
Broker
Sentiment
Analysis
Contract
{
'Positive': 0.234,
'Negative': 10.123,
'Neutral': 7.334,
'Mixed': 0
}
...
65
CloudFront
API Gateway
Authentication
You!
awslounge.onegeek.com.au
Twitter Fetch
Twitter feed
Twitter Push
IoT Message...
66
Authentication Twitter Fetch
Twitter feed
Twitter Push
IoT Message
Broker
Sentiment
Analysis
Checkpoint
#awsmelb
Challe...
WHAT WE ARE KNOWN FOR
67
Problems with automated e2e tests
● Slow
● Easy to break
● Hard to fix
● Scales badly across team...
68
C P
mock
Challenges > Async > Contract Tests
Understanding Test Symmetry
WHAT WE ARE KNOWN FOR
69
Test symmetry
Solved problems New problems
● Hard to keep both sides in
sync
● Fast feedback
● Fe...
70
C P
mock
Challenges > Async > Contract Tests
71
C P
mock
Challenges > Async > Contract Tests
PPCC
mock
contract
Challenges > Async > Contract Tests
Bug turnaround -
minutes
Contracts tests
Challenges > Async > Contract Tests
Know before you
commit
Contracts tests
Challenges > Async > Contract Tests
Make changes with
speed and confidence
Contracts tests
Challenges > Async > Contract Tests
Deploy independently
Contracts tests
Challenges > Async > Contract Tests
Better API design
(Consumer)
Contracts tests
Challenges > Async > Contract Tests
78
● Open source
● Multiple languages
○ Javascript
○ Go
○ Python
○ JVM
○ .NET
○ + more
pact.io
● HTTP contracts
● Message ...
Challenges > Async > Contract Tests
Expectations on
Sentiment Analyser
made clear
80
Acceptance Tests vs Canary Release
Challenges > Async
Summary
81
Summary
Problems
● Serverless complexity > uServices
● Testing cloud native services
● Avoiding vendor lock-in
...
Summary
82
Summary
Problems Solutions
● Microservices+
● Create cloud and protocol
anti-corruption layer
● Mock out AWS SD...
Testing large-scale, serverless
and asynchronous systems
Matt Fellows
@matthewfellows
...without end-to-end integrated tes...
Upcoming SlideShare
Loading in …5
×

Trends and development practices in Serverless architectures

180 views

Published on

AWS ISV Event - Unlocking Business Agility with the AWS Serverless Application Model

Matt Fellows, Principal Consultant from DiUS will talk about the evolution to serverless architectures, and discuss key development and testing practices for these modern distributed systems

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Trends and development practices in Serverless architectures

  1. 1. Testing large-scale, serverless and asynchronous systems Matt Fellows @matthewfellows ...without end-to-end integrated tests! #AWSAgility
  2. 2. 2 Why Serverless? https://aws.amazon.com/retail/data-analytics/
  3. 3. WHAT WE ARE KNOWN FOR 3 Microservices Evolution Drivers for levelling up ● Increase time-to-value ● Increase utilisation / cost reduction ● Raise abstraction; focus on composition ● Scale product teams ● Massively distributed, real-time apps ● API contracts (e.g. OAS) ● Ubiquity of cloud computing ● Rise of containers ● Unreliable networking / components ● Rise of the Service Mesh ● Improved observability Why Serverless? > Drivers
  4. 4. Trends
  5. 5. 5 Data Pipelines Trends > Data pipelines
  6. 6. 6 Trends > Data pipelines
  7. 7. 7 https://aws.amazon.com/blogs/big-data/automating-analytic-workflows-on-aws/ Trends > Data pipelines
  8. 8. 8 https://www.slideshare.net/nateware/aws-gdc2015-gameanalyticsfinal Trends > Data pipelines
  9. 9. 9 Trends > Data pipelines > Before Systems of Record Integration / Middleware APIs Presentation Synchronous, coupled invocation Data locked here
  10. 10. 10 Trends > Data pipelines > After System of Record Event API Local cache / materialised view API 3. Event processed into local view of data 1. Change of system state 2. Event is shared
  11. 11. Observations Architecture > Microservices ● Democratisation of data ○ “Everyone gets a data” ● Hollywood Principle ● Pipelines are the new batch ● Prerequisite for AI/ML ● Composition and enrichment ● Lambda to “glue” steps together
  12. 12. 12 IT Ops. Automation Trends > IT Ops Automation
  13. 13. Before... ● Ad-hoc ● Cron jobs running on instances ● Scripts > code ● Shared “Job” machines, with broad security permissions and “all-the-tools” ● Lots of manual “glue” and orchestration Trends > IT Ops Automation
  14. 14. 14 Trends > IT Ops Automation Decide Alarm Schedule Event subscription Notify Pagerduty Act Isolate instance from ASG Ship logs to Splunk Offsite backup Orient Security Analysis Application Anomaly / Canary Detection Policy Violation Chaos Engineering Observe Flow Logs CloudTrail CloudWatch Logs Application Event Configured Rule
  15. 15. New approach ● Scheduled Lambda to replace “cron” jobs ● Real-time cron - event-driven ● Function written and deployed as code - first class citizens Rise of the OODA loop Trends > IT Ops Automation
  16. 16. 16 The new uService Trends > Microservices
  17. 17. 17 Trends > Microservices > Before Instances Team 1 Team 2 Team 3 Team ...
  18. 18. 18 API Gateway Trends > Microservices > Before Services Services Team 1: Orders Team 2: Identity Services Team ... Team 3: Experience
  19. 19. 19 Trends > Microservices > After > New Patterns API External Events API Outside World Microservice Interface API Master Data API Inside bounded context Event Emitters API Materialised view / cache Events and Processing Data Event Handlers Processing Offline Processing
  20. 20. 20 Provisioning Amazon ML Trends > Microservices > After > Example Team 1 Bounded Context Place Order API External Events Order Status API Outside World Internal Events Microservice Interface Data, Events and Processing Suggestions API Publish Event to external subscribers Insights / Predictions Customer / Other Event Orders Push API Inside bounded context CRM Materialised view of customer Provisioning Event External Events … API
  21. 21. 21 Trends > Microservices > Evolution Sizeofthing Number of communicating things Monolith SOA Serverless Microservices Fn Luckily, built on the same primitive!
  22. 22. Observations Architecture > Microservices ● Microservices still a thing ○ ..but are now just the interface of the iceberg ● Further decomposition of services into smaller functions ● Event-driven + asynchronous > synchronous invocation ● Lambda and managed services as “glue” ● Architectural composition BUT - More things to manage
  23. 23. Summary Architecture > Microservices ● Serverless as a higher-order system ○ Composition ● IT Operations Automation ○ Better discipline (OODA) ○ Everything as code ● Microservices still a thing, new designs emerging ○ Event-driven and loosely coupled ○ “Iceberg” patterns
  24. 24. 24 Much Compose! So speed! Architecture > Functions + Serverless (Cloud Native)Architecture > Functions + Serverless (Cloud Native)
  25. 25. 25 New challenges Architecture > Functions + Serverless (Cloud Native)
  26. 26. 26 If you can’t build a good monolith you shouldn’t be doing microservices Architecture > Microservices
  27. 27. ...the µServices ride ● Ability to write a modular monolith ● Strong grasp on your domain model ● Ability to design loosely coupled services ○ = state + transactions ● Ability to scale and decouple teams ● Ability to deploy continuously ● A strong “DevOps” culture ● Be comfortable with decentralised, complex systems + uncertainty Architecture > Microservices
  28. 28. 28 If you can’t build good microservices you shouldn’t be doing serverless Architecture > Functions + Serverless (Cloud Native)
  29. 29. 29 What practices do we need in this new world? Architecture > Functions + Serverless (Cloud Native)
  30. 30. Serverless Lounge
  31. 31. 31 Demo: AWS Meetup Lounge CloudFront API Gateway Authentication You! awslounge.onegeek.com.au Twitter Fetch Twitter feed Twitter Push IoT Message Broker Sentiment Analysis Checkpoint Amazon ML #AWSAgility
  32. 32. 32 Demo: AWS Meetup Lounge Demo
  33. 33. Challenges Blasphemy!
  34. 34. 34 Smaller functions = more things! Challenges > Async
  35. 35. WHAT WE ARE KNOWN FOR 35 Options ● ... Challenges ● Observability + Tracing ● Structured logging ● Shift responsibility into platform ● Good acceptance tests Challenges > Vendor lock-in
  36. 36. 36 Vendor lock-in Challenges > Vendor lock-in
  37. 37. 37 https://twitter.com/alambert/status/958050456101642241 Challenges > Vendor lock-in
  38. 38. WHAT WE ARE KNOWN FOR 38 Options ● Multi-cloud = lowest common denominator ● There is always a vendor (even if it’s you!) ● You’ll never be completely vendor agnostic ● Multi-cloud abstraction ● Avoid altogether ● Structure your code better Challenges Challenges > Vendor lock-in
  39. 39. WHAT WE ARE KNOWN FOR 39 Options ● Multi-cloud = lowest common denominator ● There is always a vendor (even if it’s you!) ● You’ll never be completely vendor agnostic ● Multi-cloud abstraction ● Avoid altogether ● Structure your code better Challenges Challenges > Vendor lock-in
  40. 40. Lambda Handler Challenges > Vendor lock-in > Lambda Anti-pattern { "Records": [ { "EventVersion": "1.0", "EventSubscriptionArn": "somearn", "EventSource": "aws:sns", "Sns": { ... } } ] } { "Success": true }
  41. 41. MQ Handler WebSocket handler MQ Adapter WS Adapter Azure Function Adapter Google Cloud Fn Adapter Azure Function Handler Google Cloud Fn Handler Lambda Handler Lambda Adapter Service Repository Adapter (Anti-corruption layer) Ports Business Logic Collaborators ... Challenges > Vendor lock-in > Ports and Adaptors
  42. 42. 42 Challenges > Vendor lock-in > Ports and Adaptors > Sentiment Analysis Example Twitter feed Sentiment Analysis IoT Message Broker
  43. 43. Challenges > Vendor lock-in > Ports and Adaptors > Lambda Port
  44. 44. Challenges > Vendor lock-in > Ports and Adaptors > SNS Adapter
  45. 45. Challenges > Vendor lock-in > Ports and Adaptors > Lambda Sentiment Service
  46. 46. Summary Architecture > Microservices ● Separate protocol handling from business logic ● Business logic shouldn’t change with introduction of a new protocol ● Port, Adapter and Business Logic are independently testable
  47. 47. 47 How do you test locally? Challenges > Testing Locally
  48. 48. WHAT WE ARE KNOWN FOR 48 Options Challenges > Testing Locally ● Stub services (e.g. Localstack) ● Multiple handlers ● Stub API (e.g. Moto) ● Unit tests
  49. 49. 49 Challenges > Testing Locally
  50. 50. WHAT WE ARE KNOWN FOR 50 Options ● Feature arms race ● Trustworthy? ● Integration style = slower ● All cloud providers? ● Stub services (e.g. Localstack) ● Stub API (e.g. Moto) ● Multiple handlers ● Unit tests Challenges > Testing Locally Challenges
  51. 51. WHAT WE ARE KNOWN FOR 51 Writing good unit tests ● Follow test pyramid ● Stub out AWS APIs ● Test your business logic separately from any serverless features (e.g. lambda handler) ● Callback pyramid of death ● Tests are hard to write ● Your business logic knows about AWS ● You have to stub multiple services per test ● Large methods ● Confusing control flow Challenges > Testing Locally > Unit Test Code Smells
  52. 52. 52 Challenges > Testing Locally > Unit Test > Twitter Fetch Checkpoint Twitter Fetch
  53. 53. Challenges > Testing Locally > Unit Test > Lambda Service Look ma, no AWS Code
  54. 54. Challenges > Testing Locally > Unit Test > Lambda Service Look ma, no AWS Code
  55. 55. 55 Challenges > Testing Locally > Unit Test > Sentiment Analysis Example Twitter feed Sentiment Analysis
  56. 56. Integration-style test of handler Challenges > Testing Locally > Unit Test > Lambda Handler
  57. 57. Summary Architecture > Microservices ● Business logic should be tested separately from port + adapter ● Majority of tests, should be fast unit tests ● Stub AWS Services where required ● (Lambda) Handlers can be locally integration tested with pre-canned JSON events
  58. 58. 58 Async - who is my consumer? Challenges > Async
  59. 59. 59 Async - delay in feedback Challenges > Async
  60. 60. WHAT WE ARE KNOWN FOR 60 Options ● Local integration testing ● Cloud env. integration testing ● Contract testing Challenges > Async
  61. 61. WHAT WE ARE KNOWN FOR 61 Options ● Can’t run complete local environment ● Many moving parts ● Problems with integration testing... ● Local integration testing ● Cloud env. integration testing ● Contract testing Challenges Challenges > Async
  62. 62. WHAT WE ARE KNOWN FOR 62 Options ● Can’t run complete local environment ● Many moving parts ● Problems with integration testing... ● Local integration testing ● Cloud env. integration testing ● Contract testing Challenges Challenges > Async
  63. 63. 63 Contract tests Challenges > Async
  64. 64. 64 IoT Message Broker Sentiment Analysis Contract { 'Positive': 0.234, 'Negative': 10.123, 'Neutral': 7.334, 'Mixed': 0 } Consumer Provider Challenges > Async > Contract Tests
  65. 65. 65 CloudFront API Gateway Authentication You! awslounge.onegeek.com.au Twitter Fetch Twitter feed Twitter Push IoT Message Broker Sentiment Analysis Checkpoint Amazon ML #awsmelb Challenges > Async > Contract Tests
  66. 66. 66 Authentication Twitter Fetch Twitter feed Twitter Push IoT Message Broker Sentiment Analysis Checkpoint #awsmelb Challenges > Async > Contract Tests > Contracts are everywhere! Contract Contract Contract Contract Contract Contract Contract Contract
  67. 67. WHAT WE ARE KNOWN FOR 67 Problems with automated e2e tests ● Slow ● Easy to break ● Hard to fix ● Scales badly across teams ● Lots of set up maintenance ● $$ potentially costly Challenges > Async > Contract Tests
  68. 68. 68 C P mock Challenges > Async > Contract Tests Understanding Test Symmetry
  69. 69. WHAT WE ARE KNOWN FOR 69 Test symmetry Solved problems New problems ● Hard to keep both sides in sync ● Fast feedback ● Few dependencies ● No dedicated environment ● Reliable ● Easy to debug Challenges > Async > Contract Tests
  70. 70. 70 C P mock Challenges > Async > Contract Tests
  71. 71. 71 C P mock Challenges > Async > Contract Tests
  72. 72. PPCC mock contract Challenges > Async > Contract Tests
  73. 73. Bug turnaround - minutes Contracts tests Challenges > Async > Contract Tests
  74. 74. Know before you commit Contracts tests Challenges > Async > Contract Tests
  75. 75. Make changes with speed and confidence Contracts tests Challenges > Async > Contract Tests
  76. 76. Deploy independently Contracts tests Challenges > Async > Contract Tests
  77. 77. Better API design (Consumer) Contracts tests Challenges > Async > Contract Tests
  78. 78. 78 ● Open source ● Multiple languages ○ Javascript ○ Go ○ Python ○ JVM ○ .NET ○ + more pact.io ● HTTP contracts ● Message contracts ● … protocol agnostic! Challenges > Async > Contract Tests
  79. 79. Challenges > Async > Contract Tests Expectations on Sentiment Analyser made clear
  80. 80. 80 Acceptance Tests vs Canary Release Challenges > Async
  81. 81. Summary 81 Summary Problems ● Serverless complexity > uServices ● Testing cloud native services ● Avoiding vendor lock-in ● Handling events and asynchronous Fn’s ● More Fn’s = more things!
  82. 82. Summary 82 Summary Problems Solutions ● Microservices+ ● Create cloud and protocol anti-corruption layer ● Mock out AWS SDK in unit tests ● Use contract tests ● Reduce reliance on end-to-end environments ● Balance ad-hoc exploratory + acceptance tests ● Serverless complexity > uServices ● Testing cloud native services ● Avoiding vendor lock-in ● Handling events + asynchronous Fn’s ● More Fn’s = more things!
  83. 83. Testing large-scale, serverless and asynchronous systems Matt Fellows @matthewfellows ...without end-to-end integrated tests!

×