Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Become a Serverless Black Belt: Optimizing Your Serverless Applications - SRV401 - re:Invent 2017

947 views

Published on

Are you an experienced serverless developer who wants a handy guide to unleash the full power of serverless architectures for your production workloads? Do you have questions about whether to choose a stream or an API as your event source, or whether to have one function or many? In this talk, we discuss architectural best practices, optimizations, and handy little cheat codes to build secure, high-scale, high-performance serverless applications, using real customer scenarios to illustrate the benefits.

  • Be the first to comment

Become a Serverless Black Belt: Optimizing Your Serverless Applications - SRV401 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Become a Serverless Black Belt Optimizing Your S er ver less Applications A j a y N a i r , P r i n c i p a l P r o d u c t M a n a g e r , A W S S e r v e r l e s s A p p l i c a t i o n s P e t e r S b a r s k i , V P , A C l o u d G u r u S R V 4 0 1
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVERLESS CUSTOMERS
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless is about maximizing elasticity, cost savings, and agility.
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MULTIPLE POINTS TO OPTIMIZE Amazon API Gateway Amazon Alexa AWS IoT Amazon Kinesis Amazon SNS Amazon SES AWS Step Functions 2 Invocations 1 Functions 3 Interactions Amazon S3 Amazon DynamoDB Custom endpoints Amazon CloudWatch Amazon Elasticsearch EC2 instance
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. OPTIMIZATION KATAS 1. THE LEAN FUNCTION 2. EVENTFUL INVOCATIONS 3. COORDINATED CALLS 4. SERVICEFUL OPERATIONS
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GOAL TODAY Repeatable regimen for building highly resilient, high-performance serverless applications.
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE LEAN FUNCTION K A T A # 1 Concise logic, efficient/single purpose code, ephemeral environment
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ANATOMY OF A FUNCTION Your function Language runtime Function container Compute substrate
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE REQUEST LIFECYCLE Bootstrap the runtime Start your code Cold start Warm start Download your code Start new container AWS optimization Your optimization
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SAME VIEW IN X-RAY
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE FUNCTION LIFECYCLE Bootstrap the runtime Start your code Warm start Download your code Start new container AWS optimization Your optimization Cold start
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EFFICIENT FUNCTION CODE • Avoid “fat”/monolithic functions • Control the dependencies in your function's deployment package • Optimize for your language • Node – Browserfy, Minify
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. JAVA – SCOPE YOUR POM FILE <dependencyManagement> <dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-bom</artifactId> <version>2.10.10</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-s3</artifactId> <version>1.10.5</version> </dependency> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-dynamodb</artifactId> <version>1.10.10</version> </dependency> <dependencies> Maven Bill Of Materials (BOM) module for AWS SDK Avoid aws-java-sdk directly! Select service dependencies only
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EPHEMERAL FUNCTION ENVIRONMENT • Lambda processes a single event per-container • No need for non-blocking execution on the frontend • REMEMBER – containers are reused • Lazily load variables in the global scope • Don’t load it if you don’t need it – cold starts are affected import boto3 client = None def my_handler(event, context): global client if not client: client = boto3.client("s3") # process
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CONCISE FUNCTION LOGIC • Separate Lambda handler (entry point) from core logic • Use functions to TRANSFORM, not TRANSPORT • Read only what you need • Query filters in Amazon Aurora • Use Amazon S3 select (new!)
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 200 seconds and 11.2 cents # Download and process all keys for key in src_keys: response = s3_client.get_object(Bucket=src_bucket, Key=key) contents = response['Body'].read() for line in contents.split('n')[:-1]: line_count +=1 try: data = line.split(',') srcIp = data[0][:8] …. 95 seconds and costs 2.8 cents # Select IP Address and Keys for key in src_keys: response = s3_client.select_object_content (Bucket=src_bucket, Key=key, expression = SELECT SUBSTR(obj._1, 1, 8), obj._2 FROM s3object as obj) contents = response['Body'].read() for line in contents: line_count +=1 try: …. A f t e rB e f o r e SMALL CHANGES, BIG DIFFERENCE (https://github.com/awslabs/lambda-refarch-mapreduce)
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SMART RESOURCE ALLOCATION Match resource allocation (up to 3 GB!) to logic Stats for Lambda function that calculates 1000 times all prime numbers <= 1000000 128 MB 11.722965sec $0.024628 256 MB 6.678945sec $0.028035 512 MB 3.194954sec $0.026830 1024 MB 1.465984sec $0.024638
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IMPACT OF MEMORY CHANGE 50% increase in memory 95th percentile changes from 3s to 2.1s https://blog.newrelic.com/2017/06/20/lambda-functions-xray-traces- custom-serverless-metrics/
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DON’T GUESSTIMATE! alexcasalboni aws-lambda-power-tuning
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MULTITHREADING? MAYBE • <1.8GB is still single core • CPU bound workloads won’t see gains – processes share same resources • >1.8GB is muti-core • CPU bound workloads will gains, but need to multi thread • I/O bound workloads WILL likely see gains • e.g. parallel calculations to return
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. NO ORCHESTRATION IN CODESTARTJOB JOB#XSTARTED HTTPPOST HTTPPOST AREWETHEREYET? NOPE! WE’REDONE! ZzZz OR time.sleep(10)
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. NO ORCHESTRATION IN CODE
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE LEAN FUNCTION K A T A # 1 Concise logic, efficient/single purpose code, ephemeral environment
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EVENTFUL INVOCATIONS K A T A # 2 Succinct payloads, resilient routing, concurrent execution
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. INVOCATION PATHS Amazon API Gateway Amazon Alexa AWS IoT Amazon Kinesis Amazon SNS Amazon SES AWS Step Functions Amazon S3 Amazon DynamoDB Custom endpoints Amazon CloudWatch Amazon Elasticsearch EC2 instance
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GATEWAYS AND ROUTERS • Choose suitable entry point for client applications • Single, custom client? Use the AWS SDK • Not end user facing? use regional endpoints on API Gateway • Discard uninteresting events ASAP • S3 – Event prefix • SNS – Message filtering (new!)
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Scrutinize the event • Must have provenance i.e. “What happened for this notification to occur?” • Additional content – identifier or payload • Remember payload constraints • Async invocation is only 128K • Avoid large responses like an image SUCCINT INVOCATIONS
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EXAMPLE - SWITCH TO BINARY 'use strict'; const co = require('co'); const Promise = require('bluebird'); const protobuf = Promise.prmisifyAll(require("protobufjs")); const lib = require('./lib'); const fs = require('fs'); module.exports.handler = co.wrap(function* (event, context, callback) { console.log(JSON.stringify(event)); let players = lib.genPlayers(); let root = yield protobuf.loadAsync("functions/player.proto"); let Players = root.lookupType("protodemo.Players"); let message = Players.create(players); let buffer = Players.encode(message).finish(); const response = { statusCode: 200, headers: { 'Content-Type': 'application/x-protobuf' }, body: buffer.toString('base64'), isBase64Encoded: true }; http://theburningmonk.com/2017/09/using-protocol-buffers-with-api- gateway-and-aws-lambda/ The same response in Protocol Buffers is nearly 40% smaller compared to default JSON
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VS. RESILIENT: USE AN EVENT STORE
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CONCURRENCY vs LATENCY Streams • Maximum theoretical throughput: # shards * 2 MB / (s) • Effective theoretical throughput: ( # shards * batch size (MB) ) / ( function duration (s) * retries until expiry) • If put / ingestion rate is greater than the theoretical throughput, consider increasing number of shards while optimizing function duration to increase throughput Everything else • Maximum Processing rate : Maximum concurrency / average duration (events per second) • Effective Processing rate : Effective concurrency / average duration (events per second) • Use concurrency metric (new!) and duration metric to estimate processing time
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THINK CONCURRENT, NOT TPS Queue based Simple No event store Stream based
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RESILENT: RETRY POLICIES • Understand retry policies • Sync never retried • Async retried 2 times • Streams retried all the time • Leverage Dead Letter Queues • SQS or SNS for replays • REMEMBER: Retries count as invokes
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BUILD YOUR OWN http://theburningmonk.com/2017/04/aws-lambda-3-pro-tips-for-working-with-kinesis-streams/
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EVENTFUL INVOCATIONS K A T A # 2 Succinct payloads, resilient routing, concurrent execution
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. COORDINATED CALLS K A T A # 3 Decoupled via APIs, scale-matched downstream, secured
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DECOUPLED: APIs AS CONTRACTS Ingestion service Ingestion API ingest & sanitize() Metadata service CRUD API read & write metadata() Frontend service Frontend API express()
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SCALE-MATCHED: CONCURRENCY CONTROLS • Concurrency a shared pool by default • Separate using per function concurrency settings • Acts as reservation • Also acts as max concurrency per function • Especially critical for data sources like RDS • “Kill switch” – set per function concurrency to zero
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SECURED: DO I NEED A VPC? Should my Lambda function be in a VPC? Does my function need to access any specific resources in a VPC? Does it also need to access resources or services in the public internet? Don’t put the function in a VPC Put the function in a private subnet Put the function in a subnet with a NAT’d route to the internet Yes Yes No No
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Download your code Start new container Create VPC ENI Start your code Attach VPC ENI Full cold start Warm start Bootstrap runtime AWS optimization Your optimization SECURED: VPC vs LATENCY
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SECURED: VPC vs RESILIENCE • ALWAYS configure a minimum of 2 Availability Zones • Give your Lambda functions their own subnets • Give your Lambda subnets a large IP range to handle potential scale • If your functions need to talk to a resource on the internet, you need a NAT!
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. COORDINATED CALLS K A T A # 3 Decoupled via APIs, scale-matched downstream, secured
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVICEFUL OPERATIONS K A T A # 4 Automated operations, Monitored applications, Innovation mindset In the end, it’s about the people
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A CLOUD GURU: SERVERLESS COMPANY
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WEEKLY STATS 6.21M Lambda Invocations ~ 4M API Requests 480 Lambda Functions 15 Environments
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2+ TB of data in S3 7+ TB of data served weekly via CloudFront WEEKLY STATS
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOMATE MONITOR INNOVATE
  47. 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOMATION AT ACG ANSIBLE Other Services Frontend
  48. 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOMATION AT ACG: TESTING Selenium Jester
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOMATE MONITOR INNOVATE
  50. 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MONITORING - ALERTS 566 CloudWatch alarmsDashboards covering services we use
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MONITORING - DASHBOARDS Runscope Monitoring
  52. 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MONITORING - SECURITY
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOMATE MIGRATE INNOVATE
  54. 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. INCREMENTAL ARCHITECTURE Serverless Monolith Serverless Microservices
  55. 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. INNOVATION
  56. 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. INNOVATION
  57. 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A FEW PUBLIC PROJECTS FROM ACG
  58. 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless is a mindset change toward automation, agility, and innovation.
  59. 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVICEFUL OPERATIONS K A T A # 4 Automated operations, Monitored applications, Innovation mindset In the end, it’s about the people
  60. 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. OPTIMIZATION KATAS 1. THE LEAN FUNCTION CONCISE. EFFICIENT. EPHEMERAL. 2. EVENTFUL INVOCATIONS SUCCINT. RESILIENT. CONCURRENT. 3. COORDINATED CALLS DECOUPLED. SCALE MATCHED. SECURED. 4. SERVICEFUL OPERATIONS AUTOMATE. MONITOR. INNOVATE.
  61. 61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MORE OPTIMIZATION SESSIONS SRV303 - Monitoring and Troubleshooting in a Serverless World SRV322-R2 - Migration to Serverless: Design Patterns and Best Practices SRV311 - Authoring and Deploying Serverless Applications with AWS SAM SRV320-R - Best Practices for Using AWS Lambda with RDS/RDBMS Solutions
  62. 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THANK YOU!

×