Building Event-Driven Applications with Serverless and AWS - AWS Summit New York

1. S U M M I T NEW YORK

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Building event-driven applications with serverless and AWS Jeremy Daly Chief Technology Officer, AlertMe.news @jeremy_daly D E V 1 1

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T About Me • CTO at AlertMe.news • Consult with companies building in the cloud • 20+ year veteran of technology startups • Started working with AWS in 2009 and started using Lambda in 2015 • Blogger (jeremydaly.com), open-source contributor, speaker • Publish the Off-by-none serverless newsletter • Host of the Serverless Chats podcast

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Basic Serverless Workflow LAMBDA EVENT SOURCES … CLOUD RESOURCES Event Trigger Unique business logicEVENT Output

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simple Web Service Client API Gateway Lambda DynamoDB “Synchronous” Request

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simple S3 Trigger Client Lambda DynamoDBS3 Bucket “Asynchronous” Request

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What are Events? • Events are FACTS • Events are IMMUTABLE • Events are OBSERVABLE • Events are TEMPORAL Timestamp Event 2019-07-11 16:35:20 OrderCreated 2019-07-11 16:35:43 PaymentProcessed 2019-07-11 16:36:01 InventoryUpdated 2019-07-11 16:36:07 ConfirmationSent … …

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Event-Driven versus Event-Sourcing Event-Driven • Domain events, i.e. something happened within a bounded context • High-value semantics (user registered, order placed, payment processed) • Relevant within a bounded context and across bounded contexts Event-Sourcing • A stored sequence of events that describes the current state • Events are specific to a bounded context, i.e. domain logic is required to reconstruct state (provide email, verify email) • These events aren’t particularly interesting outside of the bounded context

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Communication Types Synchronous Communication Services can be invoked by other services and must wait for a reply. This is considered a blocking request, because the invoking service cannot finish executing until a response is received. Asynchronous Communication This is a non-blocking request. A service can invoke (or trigger) another service directly or it can use another type of communication channel to queue information. The service typically only needs to wait for confirmation (ack) that the request was received.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Orchestration versus Choreography 💃🕺 Orchestration • Use a controller to coordinate the behavior of multiple components or services • Uas a way to maintain state between multiple steps in a transaction Choreography • Messages are sent to an event bus or queue and are picked up by other services • We are not waiting for downstream services to complete before continuing “Each service is smart enough to understand its role in the whole dance.” ~ Sam Newman AWS Step Functions

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using SNS to Route Events Client SNSTopic “Asynchronous” Request ack Serverless Microservice HTTP SMS Lambda SQS Email Key Points: • SNS has a “well-defined API” • Decouples downstream processes • Allows multiple subscribers with message filters

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Event Forking with SNS “Asynchronous” Request ack SQS Queue LambdaSNS Topic DynamoDBClient API Gateway Lambda Event processing pipeline SQS Queue Lambda Kinesis Firehose Event Search & Analytics pipeline Elasticsearch S3 Dead Letter Bucket SQS Queue Lambda Kinesis Firehose Event Storage & Backup pipeline S3 Backup Bucket amount > $100 Key Points: • SNS to SQS is “guaranteed” (100,010 retries) • Filters allow you to control pipeline data • Common patterns that can be reused Event Fork Pipelines Available in the Serverless Application Repository Event Fork Pipelines Available in the Serverless Application Repository

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Distribute & Throttle Stripe API ack SQS Queue Lambda (concurrency 25) SNS Topic Client API Gateway Lambda Order Service total > $0 RDS SQS Queue Lambda (concurrency 10) SMS Alerting Service Twilio API SQS Queue Lambda (concurrency 5) Billing Service status == ”order_complete” Event Service Key Points: • Filter events to selectively trigger services • Manage throttling/quotas per service • Reuses the same SNS topic to consolidate events

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CloudWatch Events https://aws.amazon.com/blogs/aws/ building-serverless-pipelines-with- amazon-cloudwatch-events/

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T EventBridge has 95+ Event Sources and 17 Target Types! • Batch job queue • CloudWatch log group • CodeBuild project • CodePipeline • EC2 (CreateSnapshot, RebootInstances, StopInstances, TerminateInstances APIs) • ECS task • Event bus in another AWS account • Firehose delivery stream • Inspector assessment template • Kinesis stream • Lambda function • SNS topic • SQS queue • SSM Automation • SSM OpsItem • SSM Run Command • Step Functions state machine NO SMS, HTTP, MOBILE or EMAIL

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T SNS versus CloudWatch Events versus Kinesis • SNS subscriptions can FILTER events • Advanced pattern matching through filter policies • String and Numeric Value Matching (Whitelisting, Blacklisting, and Prefix Matching and Range Matching) – ONLY ON MESSAGE ATTRIBUTES • CloudWatch Event rules can MATCH events • Matching is exact (character-by-character) using event patterns, without case- folding or any other string normalization – ON ALL EVENT FIELDS • Kinesis Streams can do NEITHER • Event filtering must be done by the consumer

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multicasting with EventBridge Asynchronous “PutEvents” Request ack w/ event id Key Points: • Create up to 100 event buses per account • Allows multiple subscribers with RULES and EVENT PATTERNS • Forward events to other accounts Amazon EventBridge Lambda SQS Client Step Function Event Bus +13 others

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simple Serverless Webhook Client API Gateway Lambda DynamoDB Highly Scalable Highly Scalable Highly Scalable

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simple Serverless Webhook Client API Gateway Lambda Highly Scalable Highly Scalable ^ not so RDS NotThat Scalable 😳 RDBMS and FaaS don’t play nicely together: • Concurrency model doesn’t allow connection pooling • Limited number of DB connections available • Recycled containers create zombies

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scalable Serverless Webhook RDSClient API Gateway SQS Queue SQS (DLQ) Lambda Lambda (throttled) ack “Asynchronous” Request Synchronous Request

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. High Throughput Serverless Webhook RDSClient SQS Queue SQS (DLQ) Lambda Lambda (throttled) ack “Asynchronous” Request Synchronous Request ALB Lambda TargetType Use Application Load Balancer as your HTTP endpoint

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Direct Serverless Webhook RDSClient API Gateway SQS Queue SQS (DLQ) Lambda (throttled) ack “Asynchronous” Request Synchronous Request Use service integrations with VTL to bypass Lambda

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Integrating with Partners (no more webhooks) • Provide AWS Account ID to an approved Partner • Partner submits request to publish events to AWS Account • Select Partner Event Sources, approve access, and create event bus • Partner publishes events (e.g. standard webhook events) using the PutPartnerEvents API • Account creates rules and event patterns to subscribe to events and route them to targets • Partner is responsible for retries to EventBridge

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. EventBridge “Webhook” RDSSaaS Provider SQS Queue SQS (DLQ) Lambda (throttled) SaaS Provider sends events directly to an Partner specific Event Bus in your AWS account Amazon EventBridge Use rules and event patterns to subscribe consumers

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Takeaways • Events are immutable facts that can be multicast • Events minimize service coupling and increase autonomy • Think asynchronously and embrace eventual consistency • Event-driven applications should broadcast events with high-value semantics that are relevant across bounded contexts • Use the right tool for the job, and don’t reinvent the wheel (SNS, CloudWatch Events, Partner Event Buses, SQS, Event Fork Pipelines) • Use Step Functions for distributed transactions

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Jeremy Daly @jeremy_daly jeremydaly.com

Building Event-Driven Applications with Serverless and AWS - AWS Summit New York

Building Event-Driven Applications with Serverless and AWS - AWS Summit New York

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Building Event-Driven Applications with Serverless and AWS - AWS Summit New York

Similar to Building Event-Driven Applications with Serverless and AWS - AWS Summit New York(20)

Recently uploaded

Recently uploaded(20)

Building Event-Driven Applications with Serverless and AWS - AWS Summit New York