Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless Architecture Patterns

Serverless Architecture Patterns

  • Login to see the comments

Serverless Architecture Patterns

  1. 1. Serverless architectural patterns Ric Harvey, Technical Developer Evangelist @ric__Harvey
  2. 2. What does serverless mean? No servers to provision or manage Scale with your usage Built in availability and fault- tolerance Never pay for idle/unused capacity
  3. 3. Serverless runs on functions • Functions are the unit of deployment and scale • This scales per request! • Skip the boring parts, skip the hard parts
  4. 4. Serverless applications FUNCTION SERVICES (ANYTHING) Changes in data state Requests to endpoints Changes in resource state Node Python Java C# EVENT SOURCE
  5. 5. Example event sources Data stores Endpoints Configuration repositories Event/message sources Amazon S3 Amazon DynamoDB Amazon Kinesis Amazon Cognito Amazon IoT AWS Step Functions Amazon Alexa AWS CloudTrail AWS CodeCommit Amazon CloudWatch Amazon SES Amazon SNS Cron events Amazon API Gateway AWS Cloudformation …and more!
  6. 6. A few Lambda specific best practices • Lambda is stateless à architect accordingly! • Assume no affinity with underlying compute infrastructure • Local filesystem and child processes may not extend beyond the lifetime of the Lambda request
  7. 7. Lambda considerations and best practices • Can your Lambda functions survive the cold? • Instantiate AWS clients and database clients outside the scope of the handler to take advantage of connection re-use. • Schedule with CloudWatch Events for warmth • ENIs for VPC support are attached during cold start import sys import logging import rds_config import pymysql rds_host = "rds-instance" db_name = rds_config.db_name try: conn = pymysql.connect( except: logger.error("ERROR: def handler(event, context): with conn.cursor() as cur: Executes during cold start Executes with each invocation
  8. 8. Lambda considerations and best practices • How about a file system? • Don’t forget about /tmp (512 MB of scratch space) exports.ffmpeg = function(event,context) { new ffmpeg('./thumb.MP4', function (err, video) { if (!err) { video.fnExtractFrameToJPG('/tmp’) function (error, files) { … } … if (!error) console.log(files); context.done(); ...
  9. 9. Lambda considerations and best practices • Custom CloudWatch metrics • 40 KB per POST • Default Acct Limit of 150 TPS • Consider aggregating with Kinesis def put_cstate ( iid, state ): response = cwclient.put_metric_data( Namespace='AWSx/DirectConnect', MetricData=[ { 'MetricName':'ConnectionState', 'Dimensions': [ { 'Name': 'ConnectionId', 'Value': iid }, ], 'Value': state, 'Unit': 'None’
  10. 10. A couple kinds of design patterns
  11. 11. Pattern one: 3-Tier Web Application
  12. 12. 3-Tier web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3 Browser Amazon CloudFront Amazon S3 Amazon API Gateway Dynamic content in AWS Lambda Data store in Amazon DynamoDB
  13. 13. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Bucket Policies • ACLs • OAI • Geo-Restriction • Signed Cookies • Signed URLs • DDOS IAM AuthZ IAM Serverless web app security • Throttling • Caching • Usage Plans Static Content Browser Lambda@Edge
  14. 14. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Bucket Policies • ACLs • OAI • Geo-Restriction • Signed Cookies • Signed URLs • DDOS IAMAuthZ IAM Serverless web app security • Throttling • Caching • Usage Plans Static Content Browser Amazon CloudFront • HTTPS • Disable Host Header Forwarding AWS WAF
  15. 15. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Access Logs in S3 Bucket • Access Logs in S3 Bucket • CloudWatch Metrics- https://aws.amazon.com/clo udfront/reporting/ Serverless web app monitoring AWS WAF • WebACL Testing • Total Requests • Allowed/Blocked Requests by ACL logslogs • Invocations • Invocation Errors • Duration • Throttled Invocations • Latency • Throughput • Throttled Reqs • Returned Bytes • Documentation • Latency • Count • Cache Hit/Miss • 4XX/5XX Errors Streams AWS CloudTrail Static Content Browser Custom CloudWatch Metrics & Alarms
  16. 16. Serverless web app lifecycle management • AWS SAM (Serverless Application Model) - blog AWS Lambda Amazon API Gateway AWS CloudFormationAmazon S3 Amazon DynamoDB Package & Deploy Code/Packages/Swagger Serverless Template Serverless Template w/ CodeUri package deploy CI/CD Tools
  17. 17. A couple words on Amazon API Gateway • Use mock integrations • Signed URL from API Gateway for large or binary file uploads to S3 • Use request/response mapping templates for legacy apps and HTTP response codes • Asynchronous calls for Lambda > 30s
  18. 18. Pattern two: Batch processing
  19. 19. Characteristics of batch processing • Large data sets • Periodic or scheduled tasks • Extract Transform Load (ETL) jobs • Usually non-interactive and long running • Many problems fit MapReduce programming model
  20. 20. AWS Lambda: Splitter Amazon S3 Object Amazon DynamoDB: Mapper Results AWS Lambda: Mappers …. …. AWS Lambda: Reducer Amazon S3 Results Serverless batch processing
  21. 21. Best practices and things to think about • Cascade mapper functions • Lambda languages vs. SQL • Speed is directly proportional to the concurrent Lambda function limit • Use DynamoDB/ElastiCache/S3 for intermediate state of mapper functions • Lambda MapReduce Reference Architecture
  22. 22. Costs of serverless batch processing? • 200 GB normalized Google Ngram data-set • Serverless: • 1000 concurrent Lambda invocations • Processing time: 9 minutes • Cost: $7.06
  23. 23. Pattern three: stream processing
  24. 24. Characteristics of stream processing • High ingest rate • Near real-time processing (low latency from ingest to process) • Spiky traffic (lots of devices with intermittent network connections) • Message durability • Message ordering
  25. 25. Sensors Amazon Kinesis: Stream Lambda: Stream Processor S3: Final Aggregated Output Lambda: Periodic Dump to S3 CloudWatch Events: Trigger every 5 minutes S3: Intermediate Aggregated Data Lambda: Scheduled Dispatcher KPL: Producer Serverless stream processing architecture
  26. 26. Fan-out pattern • Number of Amazon Kinesis Streams shards corresponds to concurrent Lambda invocations • Trade higher throughput & lower latency vs. strict message ordering Sensors Amazon Kinesis: Stream Lambda: Dispatcher KPL: Producer Lambda: Processors Increase throughput, reduce processing latency
  27. 27. More about fan-out pattern • Keep up with peak shard capacity • 1000 records / second, OR • 1 MB / second • Consider parallel synchronous Lambda invocations • Rcoil for JS (https://github.com/sapessi/rcoil) can help • Dead letter queue to retry failed Lambda invocations
  28. 28. Some event services options Amazon Kinesis Streams Amazon SQS Amazon SNS Message Durability Up to retention period Up to retention period Retry delivery (depends on destination type) Maximum Retention Period 7 days 14 days Up to retry delivery limit Message Ordering Strict within shard Standard - Best effort FIFO – Strict within Message Group None Delivery semantics Multiple consumers per shard Multiple readers per queue (but one message is only handled by one reader at a time) Multiple subscribers per topic Scaling By throughput using Shards Automatic Automatic Iterate over messages Shard iterators No No Delivery Destination Types Kinesis Consumers SQS Readers HTTP/S, Mobile Push, SMS, Email, SQS, Lambda
  29. 29. Some serverless streaming best practices • Tune batch size when Lambda is triggered by Amazon Kinesis Streams – reduce number of Lambda invocations • Tune memory setting for your Lambda function – shorten execution time • Use KPL to batch messages and saturate Amazon Kinesis Stream capacity
  30. 30. Pattern four: automation
  31. 31. Automation characteristics • Respond to alarms or events • Periodic jobs • Auditing and Notification • Extend AWS functionality • Highly Available and scalable
  32. 32. AWS Lambda: Update Route53 Amazon CloudWatch Events: Rule Triggered Amazon EC2 Instance State Changes Amazon DynamoDB: EC2 Instance Properties Amazon Route53: Private Hosted Zone Tag: CNAME = ‘xyz.example.com’ xyz.example.com A 10.2.0.134 Automation: dynamic DNS for EC2 instances
  33. 33. AWS Lambda: Resize Images Users upload photos S3: Source Bucket S3: Destination Bucket Triggered on PUTs Automation: image thumbnail creation from S3
  34. 34. A few tips from someone who knows what he’s talking about • Serverless monolith: frameworks like Zappa or Serverless that just create a single package and route all requests to the one package • Easy to port existing applications • Works well with traditional App level logging and monitoring • Easy to keep all endpoints warm since everything is hooked up • Start with a monolith then move out individual endpoints as things break • If you’re building something greenfield you can do the managed endpoints pattern with a framework like chalice - each APIGW endpoint gets created, but it’s still a single app deployment • Then there’s 1:1 every endpoint gets its own function- makes logging and introspection a nightmare but gives extreme agility for parallel development
  35. 35. A few tips from someone who knows what he’s talking about • On the non-web app side: Glue pattern • Glue is what 99% of lambda deployments are about - taking events from one service and doing something with them in another • If services are bricks then lambda is mortar
  36. 36. Other resources • Randall <3s Lambda! • @jrhunt on Twitter • Tons of examples and projects here: https://github.com/ranman • AWS documentation: http://docs.aws.amazon.com/lambda/latest/dg/welcome.html • Tons of compute blog posts: https://aws.amazon.com/blogs/compute/category/aws-lambda/ • Lambda reference architecture: https://github.com/awslabs/lambda- refarch-webapp
  37. 37. Thanks!

×