Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1) - AWS re:Invent 2018

804 views

Published on

Serverless computing allows you to build and run applications and services without thinking about servers. Serverless applications don't require you to provision, scale, and manage any servers. However, under the hood, there is a sophisticated architecture that takes care of all the undifferentiated heavy lifting for the developer. Join Holly Mesrobian, Director of Engineering, and Marc Brooker, Senior Principal of Engineering, to learn how AWS architected one of the fastest-growing AWS services. In this session, we show you how Lambda takes care of everything required to run and scale your code with high availability

A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. A Serverless Journey: Under the Hood of AWS Lambda Marc Brooker Senior Principal Engineer Serverless S R V 4 0 9 Holly Mesrobian Director of Engineering AWS Lambda
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. processes 4,000 requests per second ingests, analyzes and stores 17+ petabytes of data per season processes half a trillion validations of stock trades daily executes 16 million requests a month processes tens of billions of data points monthly SERVERLESS AT SCALE IS THE NEW NORM API traffic to register and license more than 47 million driver records in Great Britain,
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Load Balancing
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Scaling Up and Down
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Handling Failures
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Handles Load Balancing Auto Scaling Handling Failures Security Isolation Managing Utilization (and many other things) for you
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Front End Invoke Orchestrate both synchronous and asynchronous Invokes
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Counting Service Provides a region wide view of customer concurrency to help enforce set limits
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Worker Manager Tracks container idle and busy state and schedules incoming invoke requests to available containers
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Worker Provisions a secure environment for customer code execution
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Placement Service Places sandboxes on workers to maximize packing density without impacting customer experience or cold-path latency
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Routing function traffic across hosts distributed across Availability Zone
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Lambda customer (Existing Worker, New Sandbox) Availability zone 2 Availability zone 1 Invoke Front End Invoke Front End Worker Mgr Worker Mgr Reserve Sandbox Invoke Worker Worker Worker Init
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Lambda customer (Existing Worker, Existing Sandbox) Availability zone 2 Availability zone 1 Invoke Front End Invoke Front End Worker Mgr Worker Mgr Reserve Sandbox Invoke Worker Worker Worker
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SERVERLESS CUSTOMERS
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “What took us just a few days to build using a serverless solution based on AWS Lambda would have taken us six months to build from scratch.Our CTO and the rest of the project stakeholders were really happy with how much money and time we saved.” Nitin Mahajan Executive Director for Service Engineering
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Provision function capacity when needed and releasing when not needed
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Lambda customer (New Function or Scaling Up) Availability zone 2 Availability zone 1 Invoke Front End Invoke Front End Worker Mgr Worker Mgr Reserve Sandbox Invoke Init Placement Claim Worker Worker Worker Worker
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability zone 2Availability zone 1 Placement Claim Worker Worker Worker Worker Worker Worker Placement Claim Worker Worker Worker Worker Worker Worker
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Availability zone 2Availability zone 1 Placement Return Worker Worker Worker Worker Worker Worker Placement Return Worker Worker Worker Worker Worker Worker
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Cloud Region Lambda customer Availability zone 2 Availability zone 1 Front End Front End Worker Mgr Worker Mgr Placement Return Worker Worker Worker Worker
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SERVERLESS CUSTOMERS
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Handling Host and Availability Zone failure
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. With Lambda: Always have a healthy host
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instance Instance Instance InstanceInstance Instance AWS Cloud Region Availability zone 2Availability zone 1
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instance Instance Instance InstanceInstance Instance AWS Cloud Region Availability zone 2Availability zone 1
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  35. 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  36. 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  37. 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  38. 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  39. 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code
  40. 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Sandbox Lambda Runtime Your Code One Function Many Accounts
  41. 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Virtual Devices Device Emulation Physical Devices
  42. 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS virtio drivers virtio host in Firecracker Physical Devices
  43. 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hardware Host OS Hypervisor Guest OS Virtual Devices Device Emulation Physical Devices
  44. 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  45. 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. % of Resources Doing Useful Work (vs. idle or waste)
  46. 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. With Lambda: Pay only for useful work.
  47. 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Inside Lambda: Optimize To Keep Servers Busy
  48. 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  49. 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Bad: 60% 60% 60% 60% 60% 60% 60%
  50. 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Good: 99% 99% 99% 99% 0% 0% 0% Cache Locality Ability to Autoscale
  51. 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Server Bad: Pack Server With One Workload Workload Workload Workload Workload Workload Workload
  52. 52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Server Better: Pack With Many Loads Workload Workload Workload Workload Workload Workload
  53. 53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  54. 54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  55. 55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  56. 56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Server Best: Placement Optimization Workload Workload Workload Workload Workload Workload
  57. 57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Worker Lambda Function ENI in your VPC Your VPC Local NAT
  58. 58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Worker Lambda Function ENI in your VPC Your VPC Remote NAT
  59. 59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  60. 60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Firecracker Hypervisor vs. Others ↓ ↓ ↑
  61. 61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Firecracker Unlocks Higher Utilization and Scale
  62. 62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  63. 63. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Marc Brooker – Senior Principal Engineer, Amazon Serverless Holly Mesrobian – Director of Engineering, Amazon Lambda
  64. 64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×