Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Become a Serverless Black Belt - Optimizing Your Serverless Applications - AWS Online Tech Talks

1,246 views

Published on

Learning Objectives:
- Learn how to optimize Lambda function performance and cost
- Understand how to improve cold start latency
- Best practices for event invocations and concurrency

Become a Serverless Black Belt - Optimizing Your Serverless Applications - AWS Online Tech Talks

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chris Munns – Senior Developer Advocate – AWS Serverless Become a Serverless Black Belt
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. About me: Chris Munns - munns@amazon.com, @chrismunns • Senior Developer Advocate - Serverless • New Yorker • Previously: • AWS Business Development Manager – DevOps, July ’15 - Feb ‘17 • AWS Solutions Architect Nov, 2011- Dec 2014 • Formerly on operations teams @Etsy and @Meetup • Little time at a hedge fund, Xerox and a few other startups • Rochester Institute of Technology: Applied Networking and Systems Administration ’05 • Internet infrastructure geek
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. https://secure.flickr.com/photos/mgifford/4525333972 Why are we here today?
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No servers to provision or manage Scales with usage Never pay for idle Availability and fault tolerance built in Serverless means…
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVICES (ANYTHING) Changes in data state Requests to endpoints Changes in resource state EVENT SOURCE FUNCTION Node.js Python Java C# Go Serverless applications
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless is about maximizing elasticity, cost savings, and agility.
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Multiple points to optimize Amazon API Gateway Amazon Alexa AWS IoT Amazon Kinesis Amazon SNS Amazon SES AWS Step Functions 2 Invocations 1 Functions 3 Interactions Amazon S3 Amazon DynamoDB Custom endpoints Amazon Cloudwatch Amazon Elasticsearch EC2 instance
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Optimization Katas 1. THE LEAN FUNCTION 2. EVENTFUL INVOCATIONS 3. COORDINATED CALLS 4. SERVICEFUL OPERATIONS
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Goal today Repeatable regimen for building highly resilient, high-performance serverless applications.
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE LEAN FUNCTION K A T A # 1 Concise logic, efficient/single purpose code, ephemeral environment
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Anatomy of a function Your function Language runtime Execution Environment Compute substrate
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The function lifecycle Bootstrap the runtime Start your code Full cold start Partial cold start Warm start Download your code Start new container AWS optimization Your optimization
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Same view in AWS X-Ray
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The function lifecycle Bootstrap the runtime Start your code Full cold start Partial cold start Warm start Download your code Start new container AWS optimization Your optimization
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Concise function logic • Separate Lambda handler (entry point) from core logic • Use functions to TRANSFORM, not TRANSPORT • Read only what you need. For example: • Properly indexed databases • Query filters in Aurora • Use S3 select
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Small changes, big difference 200 seconds and 11.2 cents # Download and process all keys for key in src_keys: response = s3_client.get_object(Bucket=src_bucket, Key=key) contents = response['Body'].read() for line in contents.split('n')[:-1]: line_count +=1 try: data = line.split(',') srcIp = data[0][:8] …. 95 seconds and costs 2.8 cents # Select IP Address and Keys for key in src_keys: response = s3_client.select_object_content (Bucket=src_bucket, Key=key, expression = SELECT SUBSTR(obj._1, 1, 8), obj._2 FROM s3object as obj) contents = response['Body'].read() for line in contents: line_count +=1 try: AfterBefore (https://github.com/awslabs/lambda-refarch-mapreduce)
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Efficient function code • Avoid “fat”/monolithic functions • Control the dependencies in your function's deployment package • Optimize for your language • Node – Browserfy, Minify
  18. 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Java – Scope your POM file <dependencyManagement> <dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-bom</artifactId> <version>2.10.10</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-s3</artifactId> <version>1.10.5</version> </dependency> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-dynamodb</artifactId> <version>1.10.10</version> </dependency> <dependencies> Maven Bill Of Materials (BOM) module for AWS SDK Only use what you need from the aws-java-sdk! Select service dependencies only
  19. 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ephemeral function environment • Lambda processes a single event per-container • No need for non-blocking execution on the frontend • REMEMBER – containers are reused • Lazily load variables in the global scope • Don’t load it if you don’t need it – cold starts are affected import boto3 client = None def my_handler(event, context): global client if not client: client = boto3.client("s3") # process
  20. 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Smart resource allocation Match resource allocation (up to 3 GB!) to logic Stats for Lambda function that calculates 1000 times all prime numbers <= 1000000 128 MB 11.722965sec $0.024628 256 MB 6.678945sec $0.028035 512 MB 3.194954sec $0.026830 1024 MB 1.465984sec $0.024638
  21. 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Impact of a memory change 50% increase in memory 95th percentile changes from 3s to 2.1s https://blog.newrelic.com/2017/06/20/lambda-functions-xray-traces-custom-serverless-metrics/
  22. 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Don’t guestimate alexcasalboni aws-lambda-power-tuning
  23. 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Multithreading? Maybe! • <1.8GB is still single core • CPU bound workloads won’t see gains – processes share same resources • >1.8GB is multi core • CPU bound workloads will gains, but need to multi thread • I/O bound workloads WILL likely see gains • e.g. parallel calculations to return
  24. 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No orchestration in code STARTJOB JOB#XSTARTED HTTPPOST HTTPPOST AREWETHEREYET? NOPE! WE’REDONE! ZzZz OR time.sleep(10)
  25. 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No orchestration in code
  26. 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE LEAN FUNCTION K A T A # 1 Concise logic, efficient/single purpose code, ephemeral environment
  27. 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EVENTFUL INVOCATIONS K A T A # 2 Succinct payloads, resilient routing, concurrent execution
  28. 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Invocation paths Amazon API Gateway Amazon Alexa AWS IoT Amazon Kinesis Amazon SNS Amazon SES AWS Step Functions Amazon S3 Amazon DynamoDB Custom endpoints Amazon Cloudwatch Amazon Elasticsearch EC2 instance
  29. 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Gateways and routers • Choose suitable entry point for client applications • Single, custom client? Use the AWS SDK • Not end user facing? use regional endpoints on API Gateway • Discard uninteresting events ASAP • S3 – Event prefix • SNS – Message filtering
  30. 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Succinct invocations • Scrutinize the event • Must have provenance i.e. “What happened for this notification to occur?” • Additional content – identifier or payload • Remember payload constraints • Async invocation is only 128K • Avoid large responses like an image
  31. 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. http://theburningmonk.com/2017/09/using-protocol-buffers-with-api-gateway-and-aws-lambda/ The same response in Protocol Buffers is nearly 40% smaller compared to default JSON Evaluate appropriate protocol 'use strict’; const co = require('co'); const Promise = require('bluebird'); const protobuf = Promise.prmisifyAll(require("protobufjs")); const lib = require('./lib'); const fs = require('fs’); module.exports.handler = co.wrap(function* (event, context, callback) { console.log(JSON.stringify(event)); let players = lib.genPlayers(); let root = yield protobuf.loadAsync("functions/player.proto"); let Players = root.lookupType("protodemo.Players"); let message = Players.create(players); let buffer = Players.encode(message).finish(); const response = { statusCode: 200, headers: { 'Content-Type': 'application/x-protobuf' }, body: buffer.toString('base64’), isBase64Encoded: true }; callback(null, response); });
  32. 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VS. Resilient: Use an event store
  33. 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Think concurrent, not TPS Simple No event store Queue based Stream based
  34. 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Concurrency vs. Latency Streams • Maximum theoretical throughput: # shards * 2 MB / (s) • Effective theoretical throughput: ( # shards * batch size (MB) ) / ( function duration (s) * retries until expiry) • If put / ingestion rate is greater than the theoretical throughput, consider increasing number of shards while optimizing function duration to increase throughput Everything else • Maximum Processing rate : Maximum concurrency / average duration (events per second) • Effective Processing rate : Effective concurrency / average duration (events per second) • Use concurrency metric and duration metric to estimate processing time
  35. 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resilient: Retry policies • Understand retry policies • Sync never retried • Async retried 2 times • Streams retried all the time • Leverage Dead Letter Queues • SQS or SNS for replays • REMEMBER: Retries count as invokes
  36. 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EVENTFUL INVOCATIONS K A T A # 2 Succinct payloads, resilient routing, concurrent execution
  37. 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. COORDINATED CALLS K A T A # 3 Decoupled via APIs, scale-matched downstream, secured
  38. 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ingestion service Ingestion API ingest & sanitize() Metadata service CRUD API read & write metadata() Frontend service Frontend API express() Decoupled: APIs as contracts
  39. 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scale-matched: Concurrency controls • Concurrency a shared pool by default • Separate using per function concurrency settings • Acts as reservation • Also acts as max concurrency per function • Especially critical for data sources like RDS • “Kill switch” – set per function concurrency to zero
  40. 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Secured: Do I need a VPC? Should my Lambda function be in a VPC? Does my function need to access any specific resources in a VPC? Does it also need to access resources or services in the public internet? Don’t put the function in a VPC Put the function in a private subnet Put the function in a subnet with a NAT’d route to the internet Yes Yes No No
  41. 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Secured: Function lifestyle with VPC Download your code Start new container Start your code Create VPC ENI Attach VPC ENI Full cold start Warm start Bootstrap runtime Partial cold start AWS optimization Your optimization
  42. 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Secured: VPC vs. Resilience • ALWAYS configure a minimum of 2 Availability Zones • Give your Lambda functions their own subnets • Give your Lambda subnets a large IP range to handle potential scale • If your functions need to talk to a resource on the internet, you need a NAT!
  43. 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. COORDINATED CALLS K A T A # 3 Decoupled via APIs, scale-matched downstream, secured
  44. 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVICEFUL OPERATIONS K A T A # 4 Automated operations, Monitored applications, Innovation mindset In the end, it’s about the people
  45. 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automate development: Start with a framework Chalice
  46. 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ClaudiaJS Node.js framework for deploying projects to AWS Lambda and Amazon API Gateway • Has sub projects for microservices, chat bots and APIs • Simplified deployment with a single command • Use standard NPM packages, no need to learn swagger • Manage multiple versions https://claudiajs.com https://github.com/claudiajs/claudia app.js: var ApiBuilder = require('claudia-api- builder') var api = new ApiBuilder(); module.exports = api; api.get('/hello', function () { return 'hello world'; }); $ claudia create --region us-east-1 --api-module app
  47. 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chalice Python serverless “microframework” for AWS Lambda and Amazon API Gateway • A command line tool for creating, deploying, and managing your app • A familiar and easy to use API for declaring views in python code • Automatic Amazon IAM policy generation https://github.com/aws/chalice https://chalice.readthedocs.io app.py: from chalice import Chalice app = Chalice(app_name="helloworld") @app.route("/") def index(): return {"hello": "world"} $chalice deploy Chalice
  48. 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. from chalice import Chalice from chalice import BadRequestError app = Chalice(app_name='apiworld-hot') FOOD_STOCK = { 'hamburger': 'yes’, 'hotdog': 'no' } @app.route('/') def index(): return {'hello': 'world'} @app.route('/list_foods') def list_foods(): return FOOD_STOCK.keys() @app.route('/check_stock/{food}') def check_stock(food): try: return {'in_stock': FOOD_STOCK[food]} except KeyError: raise BadRequestError("Unknown food '%s', valid choices are: %s" % (food, ', '.join(FOOD_STOCK.keys()))) @app.route('/add_food/{food}', methods=['PUT']) def add_food(food): return {"value": food} Chalice – a bit deeper Chalice application routes error handling http method support
  49. 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Meet SAM!
  50. 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Serverless Application Model (SAM) CloudFormation extension optimized for serverless New serverless resource types: functions, APIs, and tables Supports anything CloudFormation supports Open specification (Apache 2.0) https://github.com/awslabs/serverless-application-model
  51. 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automated operations: Deployment Source Source CodeCommit MyApplication Build test-build-source CodeBuild Deploy Testing create-changeset AWS CloudFormation execute-changeset AWS CloudFormation Run-stubs AWS Lambda Deploy Staging create-changeset AWS CloudFormation execute-changeset AWS CloudFormation Run-API-test Runscope QA-Sign-off Manual Approval Review Deploy Prod create-changeset AWS CloudFormation execute-changeset AWS CloudFormation Post-Deploy-Slack AWS Lambda This CodePipeline pipeline: • Five Stages • Builds code artifact w/ CodeBuild • Three deployed to “Environments” • Uses SAM/CloudFormation to deploy artifact and other AWS resources • Has Lambda custom actions for running my own testing functions • Integrates with a 3rd party tool/service • Has a manual approval before deploying to production
  52. 52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Metrics • 7 Built in metrics for Lambda • Can call “put-metric-data” from your function code for custom metrics • 7 Built in metrics for API- Gateway Monitored: Metrics and logging are a universal right! CloudWatch Logs • Lambda Logging • Custom logging from your code with your language’s equivalent of console.log() • API Gateway Logging • Custom formats • Log Pivots • Build metrics based on log filters
  53. 53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  54. 54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS X-Ray
  55. 55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Application instrumentation (Node.js)
  56. 56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  57. 57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  58. 58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Innovate: Incremental infrastructure Serverless Monolith Serverless Microservices
  59. 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Innovate: Get creative!
  60. 60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SERVICEFUL OPERATIONS K A T A # 4 Automated operations, Monitored applications, Innovation mindset In the end, it’s about the people
  61. 61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Optimization Katas 1. THE LEAN FUNCTION – CONCISE. EFFICIENT. EPHEMERAL. 2. EVENTFUL INVOCATIONS – SUCCINT. RESILIENT. CONCURRENT. 3. COORDINATED CALLS – DECOUPLED. SCALE MATCHED. SECURED. 4. SERVICEFUL OPERATIONS – AUTOMATE. MONITOR. INNOVATE.
  62. 62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. aws.amazon.com/serverless
  63. 63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chris Munns munns@amazon.com @chrismunnshttps://www.flickr.com/photos/theredproject/3302110152/
  64. 64. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ? https://secure.flickr.com/photos/dullhunk/202872717/

×