Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless Architecture Patterns


Published on

As serverless architectures become more popular, AWS customers need a framework of patterns to help them deploy their workloads without managing servers or operating systems.

Serverless Architecture Patterns

  1. 1. Serverless architectural patterns Abby Fuller, AWS @abbyfuller
  2. 2. What does serverless mean? No servers to provision or manage Scale with your usage Built in availability and fault- tolerance Never pay for idle/unused capacity
  3. 3. Serverless runs on functions • Functions are the unit of deployment and scale • This scales per request! • Skip the boring parts, skip the hard parts
  4. 4. Serverless applications FUNCTION SERVICES (ANYTHING) Changes in data state Requests to endpoints Changes in resource state Node Python Java C# EVENT SOURCE
  5. 5. Example event sources Data stores Endpoints Configuration repositories Event/message sources Amazon S3 Amazon DynamoDB Amazon Kinesis Amazon Cognito Amazon IoT AWS Step Functions Amazon Alexa AWS CloudTrail AWS CodeCommit Amazon CloudWatch Amazon SES Amazon SNS Cron events Amazon API Gateway AWS Cloudformation …and more!
  6. 6. A few Lambda specific best practices • Lambda is stateless  architect accordingly! • Assume no affinity with underlying compute infrastructure • Local filesystem and child processes may not extend beyond the lifetime of the Lambda request
  7. 7. Lambda considerations and best practices • Can your Lambda functions survive the cold? • Instantiate AWS clients and database clients outside the scope of the handler to take advantage of connection re-use. • Schedule with CloudWatch Events for warmth • ENIs for VPC support are attached during cold start import sys import logging import rds_config import pymysql rds_host = "rds-instance" db_name = rds_config.db_name try: conn = pymysql.connect( except: logger.error("ERROR: def handler(event, context): with conn.cursor() as cur: Executes during cold start Executes with each invocation
  8. 8. Lambda considerations and best practices • How about a file system? • Don’t forget about /tmp (512 MB of scratch space) exports.ffmpeg = function(event,context) { new ffmpeg('./thumb.MP4', function (err, video) { if (!err) { video.fnExtractFrameToJPG('/tmp’) function (error, files) { … } … if (!error) console.log(files); context.done(); ...
  9. 9. Lambda considerations and best practices • Custom CloudWatch metrics • 40 KB per POST • Default Acct Limit of 150 TPS • Consider aggregating with Kinesis def put_cstate ( iid, state ): response = cwclient.put_metric_data( Namespace='AWSx/DirectConnect', MetricData=[ { 'MetricName':'ConnectionState', 'Dimensions': [ { 'Name': 'ConnectionId', 'Value': iid }, ], 'Value': state, 'Unit': 'None’
  10. 10. A couple kinds of design patterns
  11. 11. Pattern one: 3-Tier Web Application
  12. 12. 3-Tier web application Browser Amazon CloudFront Amazon S3 Amazon API Gateway Dynamic content in AWS Lambda Data store in Amazon DynamoDB
  13. 13. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Bucket Policies • ACLs • OAI • Geo-Restriction • Signed Cookies • Signed URLs • DDOS IAM AuthZ IAM Serverless web app security • Throttling • Caching • Usage Plans Browser
  14. 14. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Bucket Policies • ACLs • OAI • Geo-Restriction • Signed Cookies • Signed URLs • DDOS IAMAuthZ IAM Serverless web app security • Throttling • Caching • Usage Plans Browser Amazon CloudFront • HTTPS • Disable Host Header Forwarding AWS WAF
  15. 15. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Access Logs in S3 Bucket • Access Logs in S3 Bucket • CloudWatch Metrics- udfront/reporting/ Serverless web app monitoring AWS WAF • WebACL Testing • Total Requests • Allowed/Blocked Requests by ACL logslogs • Invocations • Invocation Errors • Duration • Throttled Invocations • Latency • Throughput • Throttled Reqs • Returned Bytes • Documentation • Latency • Count • Cache Hit/Miss • 4XX/5XX Errors Streams AWS CloudTrail Browser Custom CloudWatch Metrics & Alarms
  16. 16. Serverless web app lifecycle management • AWS SAM (Serverless Application Model) - blog AWS Lambda Amazon API Gateway AWS CloudFormationAmazon S3 Amazon DynamoDB Package & Deploy Code/Packages/Swagger Serverless Template Serverless Template w/ CodeUri package deploy CI/CD Tools
  17. 17. A couple words on Amazon API Gateway • Use mock integrations • Signed URL from API Gateway for large or binary file uploads to S3 • Use request/response mapping templates for legacy apps and HTTP response codes • Asynchronous calls for Lambda > 30s
  18. 18. Root/ /{proxy+} ANY Node.js Express app • Simple yet very powerful: • Automatically scale to meet demand • Only pay for the requests you receive Greedy variable, ANY method, proxy integration
  19. 19. Pattern two: Batch processing
  20. 20. Characteristics of batch processing • Large data sets • Periodic or scheduled tasks • Extract Transform Load (ETL) jobs • Usually non-interactive and long running • Many problems fit MapReduce programming model
  21. 21. AWS Lambda: Splitter Amazon S3 Object Amazon DynamoDB: Mapper Results AWS Lambda: Mappers …. …. AWS Lambda: Reducer Amazon S3 Results Serverless batch processing
  22. 22. Best practices and things to think about • Cascade mapper functions • Lambda languages vs. SQL • Speed is directly proportional to the concurrent Lambda function limit • Use DynamoDB/ElastiCache/S3 for intermediate state of mapper functions • Lambda MapReduce Reference Architecture
  23. 23. Costs of serverless batch processing? • 200 GB normalized Google Ngram data-set • Serverless: • 1000 concurrent Lambda invocations • Processing time: 9 minutes • Cost: $7.06
  24. 24. Pattern three: stream processing
  25. 25. Characteristics of stream processing • High ingest rate • Near real-time processing (low latency from ingest to process) • Spiky traffic (lots of devices with intermittent network connections) • Message durability • Message ordering
  26. 26. Sensors Amazon Kinesis: Stream Lambda: Stream Processor S3: Final Aggregated Output Lambda: Periodic Dump to S3 CloudWatch Events: Trigger every 5 minutes S3: Intermediate Aggregated Data Lambda: Scheduled Dispatcher KPL: Producer Serverless stream processing architecture
  27. 27. Fan-out pattern • Number of Amazon Kinesis Streams shards corresponds to concurrent Lambda invocations • Trade higher throughput & lower latency vs. strict message ordering Sensors Amazon Kinesis: Stream Lambda: Dispatcher KPL: Producer Lambda: Processors Increase throughput, reduce processing latency
  28. 28. More about fan-out pattern • Keep up with peak shard capacity • 1000 records / second, OR • 1 MB / second • Consider parallel synchronous Lambda invocations • Rcoil for JS ( can help • Dead letter queue to retry failed Lambda invocations
  29. 29. Amazon Kinesis Analytics Sensors Amazon Kinesis: Stream Amazon Kinesis Analytics: Window Aggregation Amazon Kinesis Streams Producer S3: Aggregated Output • CREATE OR REPLACE PUMP "STREAM_PUMP" AS INSERT INTO "DESTINATION_SQL_STREAM" • SELECT STREAM "device_id", • FLOOR("SOURCE_SQL_STREAM_001".ROWTIME TO MINUTE) as "round_ts", • SUM("measurement") as "sample_sum", • COUNT(*) AS "sample_count" • FROM "SOURCE_SQL_STREAM_001" • GROUP BY "device_id", FLOOR("SOURCE_SQL_STREAM_001".ROWTIME TO MINUTE); Aggregation Time Window
  30. 30. Some event services options Amazon Kinesis Streams Amazon SQS Amazon SNS Message Durability Up to retention period Up to retention period Retry delivery (depends on destination type) Maximum Retention Period 7 days 14 days Up to retry delivery limit Message Ordering Strict within shard Standard - Best effort FIFO – Strict within Message Group None Delivery semantics Multiple consumers per shard Multiple readers per queue (but one message is only handled by one reader at a time) Multiple subscribers per topic Scaling By throughput using Shards Automatic Automatic Iterate over messages Shard iterators No No Delivery Destination Types Kinesis Consumers SQS Readers HTTP/S, Mobile Push, SMS, Email, SQS, Lambda
  31. 31. Some serverless streaming best practices • Tune batch size when Lambda is triggered by Amazon Kinesis Streams – reduce number of Lambda invocations • Tune memory setting for your Lambda function – shorten execution time • Use KPL to batch messages and saturate Amazon Kinesis Stream capacity
  32. 32. Pattern four: automation
  33. 33. Automation characteristics • Respond to alarms or events • Periodic jobs • Auditing and Notification • Extend AWS functionality • Highly Available and scalable
  34. 34. AWS Lambda: Update Route53 Amazon CloudWatch Events: Rule Triggered Amazon EC2 Instance State Changes Amazon DynamoDB: EC2 Instance Properties Amazon Route53: Private Hosted Zone Tag: CNAME = ‘’ A Automation: dynamic DNS for EC2 instances
  35. 35. AWS Lambda: Resize Images Users upload photos S3: Source Bucket S3: Destination Bucket Triggered on PUTs Automation: image thumbnail creation from S3
  36. 36. And now, some words from a special guest.
  37. 37. Local Lambda celebrity: Randall
  38. 38. But seriously, a few tips from someone who knows what he’s talking about • Serverless monolith: frameworks like Zappa or Serverless that just create a single package and route all requests to the one package • Easy to port existing applications • Works well with traditional App level logging and monitoring • Easy to keep all endpoints warm since everything is hooked up • Start with a monolith then move out individual endpoints as things break • If you’re building something greenfield you can do the managed endpoints pattern with a framework like chalice - each APIGW endpoint gets created, but it’s still a single app deployment • Then there’s 1:1 every endpoint gets its own function- makes logging and introspection a nightmare but gives extreme agility for parallel development
  39. 39. But seriously, a few tips from someone who knows what he’s talking about • On the non-web app side: Glue pattern • Glue is what 99% of lambda deployments are about - taking events from one service and doing something with them in another • If services are bricks then lambda is mortar
  40. 40. Other resources • Randall <3s Lambda! • @jrhunt on Twitter • Tons of examples and projects here: • AWS documentation: • Tons of compute blog posts: • Lambda reference architecture: refarch-webapp
  41. 41. Thanks!