Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to improve lambda cold starts

AWS has improved Lambda cold starts by leaps and bounds in the last year. But for performance-sensitive applications such as user-facing APIs, Lambda cold starts are still a thorn in one’s side, especially when working with languages such as Java and .Net Core.

In this webinar, we will dive into strategies for improving cold start latency and how to mitigate them altogether with Provisioned Concurrency, and how Lumigo helps you optimize your use of Provisioned Concurrency.

  • Be the first to comment

How to improve lambda cold starts

  1. 1. Improving AWS Lambda Cold Starts Yan Cui @theburningmonk Developer Advocate, Lumigo AWS Serverless Hero Author of Production-Ready Serverless
  2. 2. Lambda autoscales by traffic
  3. 3. Multi-AZ by default
  4. 4. MyApiFunction Worker Worker …
  5. 5. Overloaded servers are a thing of the past
  6. 6. “cold starts”
  7. 7. www.youtube.com/watch?v=xmacMfbrG28
  8. 8. cold start
  9. 9. find space allocate worker init function module invoke function handler Time Cold Start
  10. 10. find space allocate worker init function module invoke function handler Time Cold Start out of your control
  11. 11. find space allocate worker init function module invoke function handler Time Cold Start optimize this
  12. 12. Q: When should I optimize for cold start time?
  13. 13. Q: When should I optimize for cold start time? A: When someone is going to notice it
  14. 14. Background data processing? Don’t bother.
  15. 15. Web app with stable traffic?
  16. 16. Workers are reused as much as possible.
  17. 17. time invocation
  18. 18. time invocation invocation
  19. 19. time invocation invocation
  20. 20. time invocation invocation invocation invocation
  21. 21. time invocation invocation invocation invocation invocation invocation
  22. 22. time invocation invocation invocation invocation invocation invocation
  23. 23. time invocation invocation invocation invocation invocation invocation invocation
  24. 24. time invocation invocation invocation invocation invocation invocation invocation invocation
  25. 25. Workers are garbage collected after *5-10 mins of idle. * variable
  26. 26. time invocation invocation invocation invocation invocation invocation invocation invocation
  27. 27. time invocation invocation invocation invocation invocation invocation invocation invocation
  28. 28. traffic keeps workers warm
  29. 29. Web app with stable traffic? It depends.
  30. 30. function-a function-b function-c
  31. 31. function-a function-b function-c cold startzzz!
  32. 32. It depends. What does the data say? Web app with stable traffic?
  33. 33. optimize these
  34. 34. find space allocate worker init function module invoke function handler Time Cold Start optimize this
  35. 35. User behaviours change. Traffic patterns change. Function behaviours change. Cold starts change.
  36. 36. platform.lumigo.io/signup
  37. 37. trace 500K invocations per month for FREE with promo code Yan500 platform.lumigo.io/signup
  38. 38. Yan Cui http://theburningmonk.com @theburningmonk Developer Advocate @ AWS user since 2009
  39. 39. Yan Cui http://theburningmonk.com @theburningmonk Independent Consultant advisetraining delivery
  40. 40. FREQUENCY DURATION
  41. 41. Either stop cold starts from happening, or make them fast enough it doesn’t matter anyway
  42. 42. FREQUENCY DURATION dictated by user traffic
  43. 43. Either stop cold starts from happening, or make them fast enough it doesn’t matter anyway
  44. 44. Lambda Warmers
  45. 45. time invocation invocation ping invocation invocation invocation ping ping
  46. 46. Lambda warmers are ineffective when you have more than one worker
  47. 47. pay for uptime of warm workers
  48. 48. Either stop cold starts from happening, or make them fast enough it doesn’t matter anyway
  49. 49. Q: how do I improve cold start time?
  50. 50. “use higher memory size”
  51. 51. “use higher memory size”
  52. 52. find space allocate worker init function module invoke function handler Time Cold Start Always run at FULL power!
  53. 53. find space allocate worker init function module invoke function handler Time Cold Start Always run at FULL power! EXCEPT for Provisioned Concurrency
  54. 54. https://bit.ly/32YoIbC
  55. 55. https://mikhail.io/serverless/coldstarts/aws/instances/
  56. 56. https://mikhail.io/serverless/coldstarts/aws/instances/
  57. 57. “use higher memory size” unless you’re using DotNetCore
  58. 58. More memory More expensive
  59. 59. “keep deployment artefact small”
  60. 60. find space allocate worker init function module invoke function handler Roundtrip time for cold starts
  61. 61. adds 30-60ms
  62. 62. adds up to 450ms!
  63. 63. “keep deployment artefact small”
  64. 64. doesn’t do anything with the AWS SDK
  65. 65. 383ms 391ms 402ms 408ms 444ms
  66. 66. 383ms 391ms 402ms 408ms 444ms bundling AWS SDK adds 30-60ms
  67. 67. doesn’t do anything with the AWS SDK module resolution (CPU) + Filesystem calls
  68. 68. loading from Lambda Layer is faster than bundling
  69. 69. loading from Lambda Layer is faster than bundling loading from the built-in AWS SDK is even faster
  70. 70. use the built-in AWS SDK* * AWS might update the version without notice
  71. 71. “What about the other dependencies? Put them in Lambda Layers?”
  72. 72. sure, but, Lambda Layer has problems…
  73. 73. still need dependencies for local tests
  74. 74. you need 2 deployments instead of 1… 1 for the layer, and 1 for the service
  75. 75. no semantic versioning
  76. 76. doesn’t work with compiled languages
  77. 77. [Python] use Lambda Layers
  78. 78. www.npmjs.com/package/serverless-layers
  79. 79. [Node] use Webpack
  80. 80. Treeshaking FTW!
  81. 81. doesn’t do anything with the AWS SDK module resolution (CPU) + Filesystem calls
  82. 82. require specific clients
  83. 83. saves 200ms in at least 75% of cases
  84. 84. webpack still FTW!
  85. 85. [Node] use Webpack [Node] require only clients you need
  86. 86. [Node] use Webpack [Node] require only clients you need [Node] package functions individually
  87. 87. “what about Java?”
  88. 88. [Java] one JAR per function
  89. 89. [Java] one JAR per function [Java] reduce dependencies
  90. 90. [Java] one JAR per function [Java] reduce dependencies [Java] avoid big frameworks like SpringBoot
  91. 91. [Java] one JAR per function [Java] reduce dependencies [Java] avoid big frameworks like SpringBoot [Java] use AWS SDK v2
  92. 92. [Java] one JAR per function [Java] reduce dependencies [Java] avoid big frameworks like SpringBoot [Java] use AWS SDK v2 [Java] exclude dependencies you don’t need
  93. 93. [Java] one JAR per function [Java] reduce dependencies [Java] avoid big frameworks like SpringBoot [Java] use AWS SDK v2 [Java] exclude dependencies you don’t need [Java] remove reflections
  94. 94. [Java] one JAR per function [Java] reduce dependencies [Java] avoid big frameworks like SpringBoot [Java] use AWS SDK v2 [Java] exclude dependencies you don’t need [Java] remove reflections [Java] avoid dependency injection frameworks
  95. 95. www.npmjs.com/package/lumigo-cli
  96. 96. Either stop cold starts from happening, or make them fast enough it doesn’t matter anyway
  97. 97. Doens’t work on $LATEST alias
  98. 98. Integrates with Application Auto-Scaling
  99. 99. time req/s lunch dinner Provisioned Concurrency
  100. 100. time req/s lunch dinner Provisioned Concurrency On-Demand Concurrency
  101. 101. https://lumigo.io/blog/provisioned-concurrency-the-end-of-cold-starts/
  102. 102. there are no silver bullets
  103. 103. reserved concurrency is a powerful tool IFF you have a cold start problem don’t use it by default
  104. 104. trace 500K invocations per month for FREE with promo code Yan500 platform.lumigo.io/signup
  105. 105. @theburningmonk theburningmonk.com github.com/theburningmonk yan@lumigo.io

    Be the first to comment

  • matiskay

    Sep. 24, 2020
  • kkbava

    Sep. 29, 2020

AWS has improved Lambda cold starts by leaps and bounds in the last year. But for performance-sensitive applications such as user-facing APIs, Lambda cold starts are still a thorn in one’s side, especially when working with languages such as Java and .Net Core. In this webinar, we will dive into strategies for improving cold start latency and how to mitigate them altogether with Provisioned Concurrency, and how Lumigo helps you optimize your use of Provisioned Concurrency.

Views

Total views

723

On Slideshare

0

From embeds

0

Number of embeds

18

Actions

Downloads

5

Shares

0

Comments

0

Likes

2

×