Successfully reported this slideshow.
Your SlideShare is downloading. ×

Serverless observability - a hero's perspective

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 118 Ad

Serverless observability - a hero's perspective

Download to read offline

Yan Cui, an AWS Serverless Hero, will talk about the learnings from using serverless at scale.

He will cover the challenges for observability in serverless asynchronous workloads and the patterns to address those challenges, like using centralized logging, correlation IDs, tracing, lambda extensions.

Yan Cui, an AWS Serverless Hero, will talk about the learnings from using serverless at scale.

He will cover the challenges for observability in serverless asynchronous workloads and the patterns to address those challenges, like using centralized logging, correlation IDs, tracing, lambda extensions.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Serverless observability - a hero's perspective (20)

Advertisement
Advertisement

Serverless observability - a hero's perspective

  1. 1. Serverless OBSERVABILITY A hero’s perspective
  2. 2. @theburningmonk theburningmonk.com What do I mean by “observability”?
  3. 3. Monitoring watching out for known failure modes in the system, e.g. network I/O, CPU, memory usage, …
  4. 4. @theburningmonk theburningmonk.com Observability being able to debug the system, and gain insights into the system’s behaviour
  5. 5. @theburningmonk theburningmonk.com In control theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs. https://en.wikipedia.org/wiki/Observability
  6. 6. @theburningmonk theburningmonk.com In control theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs. https://en.wikipedia.org/wiki/Observability including non- functional outputs
  7. 7. @theburningmonk theburningmonk.com These are the four pillars of the Observability Engineering team’s charter: • Monitoring • Alerting/Visualization • Distributed systems tracing infrastructure • Log aggregation/analytics “ ” http://bit.ly/2DnjyuW - Observability Engineering at Twitter
  8. 8. @theburningmonk theburningmonk.com microservices death stars circa 2015
  9. 9. @theburningmonk theburningmonk.com microservices death stars circa 2015 mm… I wonder what’s going on here…
  10. 10. @theburningmonk theburningmonk.com microservices death stars circa 2015 I got this!
  11. 11. @theburningmonk theburningmonk.com
  12. 12. Yan Cui http://theburningmonk.com @theburningmonk AWS user for 10 years
  13. 13. http://bit.ly/yubl-serverless
  14. 14. Yan Cui http://theburningmonk.com @theburningmonk Developer Advocate @
  15. 15. Yan Cui http://theburningmonk.com @theburningmonk Independent Consultant advise training delivery
  16. 16. theburningmonk.com/courses
  17. 17. Learn GraphQL and AppSync by building a Twitter clone with these technologies appsyncmasterclass.com
  18. 18. realworldserverless.com
  19. 19. @theburningmonk theburningmonk.com
  20. 20. @theburningmonk theburningmonk.com Concurrency is in your application
  21. 21. @theburningmonk theburningmonk.com Concurrency is in your application Agent
  22. 22. @theburningmonk theburningmonk.com Concurrency is in your application Agent
  23. 23. @theburningmonk theburningmonk.com Concurrency is in your application Agent Shares the same CPU, memory and bandwidth as application
  24. 24. @theburningmonk theburningmonk.com API Gateway
  25. 25. @theburningmonk theburningmonk.com API Gateway Lambda
  26. 26. @theburningmonk theburningmonk.com API Gateway Lambda Worker Worker Worker Worker Worker Worker Worker Worker Worker
  27. 27. @theburningmonk theburningmonk.com API Gateway Lambda Worker Worker Worker Worker Worker Worker Worker Worker Worker Busy Busy Busy Busy Busy Busy
  28. 28. @theburningmonk theburningmonk.com API Gateway Lambda Worker Worker Worker Worker Worker Worker Worker Worker Worker Busy Busy Busy Busy Busy Busy
  29. 29. @theburningmonk theburningmonk.com API Gateway Lambda Worker Worker Worker Worker Worker Worker Worker Worker Worker Busy Busy Busy Busy Busy Busy Only processes one request at a time
  30. 30. @theburningmonk theburningmonk.com API Gateway Lambda Worker Worker Worker Worker Worker Worker Worker Worker Worker Busy Busy Busy Busy Busy Busy Only processes one request at a time Runs in an AWS-managed baremetal EC2 instance
  31. 31. @theburningmonk theburningmonk.com Resource overhead is irrelevant. Think latency overhead instead.
  32. 32. @theburningmonk theburningmonk.com Resource overhead is irrelevant. Think latency overhead instead.
  33. 33. @theburningmonk theburningmonk.com https://www.youtube.com/watch?v=xmacMfbrG28
  34. 34. @theburningmonk theburningmonk.com Built-in collection of: Metrics Logs Traces CloudWatch Metrics CloudWatch Logs X-Ray
  35. 35. challenges
  36. 36. Background Processing Time
  37. 37. @theburningmonk theburningmonk.com user request user request user request user request user request user request user request critical paths: minimise user-facing latency handler handler handler handler handler handler handler
  38. 38. @theburningmonk theburningmonk.com user request user request user request user request user request user request user request critical paths: minimise user-facing latency StatsD handler handler handler handler handler handler handler rsyslog background processing: batched, asynchronous, low overhead
  39. 39. @theburningmonk theburningmonk.com No background processing*.
  40. 40. @theburningmonk theburningmonk.com No background processing*. * you can work around this with Lambda extensions
  41. 41. (potential) Data Loss When workers are garbage collected
  42. 42. @theburningmonk theburningmonk.com Lambda cold start
  43. 43. @theburningmonk theburningmonk.com Lambda data is batched between invocations
  44. 44. @theburningmonk theburningmonk.com Lambda idle data is batched between invocations
  45. 45. @theburningmonk theburningmonk.com Lambda idle garbage collection data is batched between invocations
  46. 46. @theburningmonk theburningmonk.com Lambda
  47. 47. @theburningmonk theburningmonk.com my code send metrics
  48. 48. @theburningmonk theburningmonk.com my code send metrics
  49. 49. @theburningmonk theburningmonk.com my code send metrics internet internet press button something happens
  50. 50. Lambda Extension to the rescue!
  51. 51. Lambda Extension to the rescue! Think “sidecar” but for Lambda ;-)
  52. 52. @theburningmonk theburningmonk.com Your code Time Invoke Idle Shutdown Runtime shutdown
  53. 53. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Runtime shutdown Shutdown
  54. 54. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Runtime shutdown Shutdown
  55. 55. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown
  56. 56. @theburningmonk theburningmonk.com Lambda idle garbage collection data is batched between invocations
  57. 57. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown Flush buffered telemetry data
  58. 58. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown 5-10 mins on average
  59. 59. @theburningmonk theburningmonk.com my code send metrics
  60. 60. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown
  61. 61. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown You pay for this time too
  62. 62. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown Return function response
  63. 63. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown Return function response Send telemetry data
  64. 64. Lots of async invocations
  65. 65. @theburningmonk theburningmonk.com http://bit.ly/2Dpidje
  66. 66. @theburningmonk theburningmonk.com ? functions are often chained together via asynchronous invocations
  67. 67. @theburningmonk theburningmonk.com ? SNS Kinesis EventBridge CloudWatch Logs IoT DynamoDB S3 SES
  68. 68. @theburningmonk theburningmonk.com S3 Lambda
  69. 69. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda
  70. 70. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda
  71. 71. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda
  72. 72. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda
  73. 73. @theburningmonk theburningmonk.com Tracing through asynchronous invocations can be difficult
  74. 74. @theburningmonk theburningmonk.com Use correlation IDs to find related logs
  75. 75. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda How to inject correlation IDs?
  76. 76. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda How to inject correlation IDs? How to inject correlation IDs? How to inject correlation IDs?
  77. 77. @theburningmonk theburningmonk.com https://github.com/getndazn/dazn-lambda-powertools
  78. 78. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda
  79. 79. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda Microservice 1 Microservice 2 Microservice 3
  80. 80. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda Microservice 1 Microservice 2 Microservice 3 AWS account 1 AWS account 2 AWS account 3
  81. 81. Log Aggregation
  82. 82. @theburningmonk theburningmonk.com
  83. 83. @theburningmonk theburningmonk.com
  84. 84. @theburningmonk theburningmonk.com
  85. 85. @theburningmonk theburningmonk.com CloudWatch Logs is an async event source for Lambda
  86. 86. @theburningmonk theburningmonk.com Concurrent Executions Time regional max concurrency functions that are delivering business value
  87. 87. @theburningmonk theburningmonk.com Concurrent Executions Time regional max concurrency functions that are delivering business value ship logs
  88. 88. @theburningmonk theburningmonk.com either set concurrency limit on the log shipping function (and potentially lose logs due to throttling) or…
  89. 89. @theburningmonk theburningmonk.com
  90. 90. @theburningmonk theburningmonk.com 1 shard = 1 concurrent execution i.e. control the no. of concurrent executions with no. of shards
  91. 91. @theburningmonk theburningmonk.com or consider using Lambda Extensions
  92. 92. @theburningmonk theburningmonk.com or consider using Lambda Extensions Optionally disable CloudWatch Logs to save on cost
  93. 93. @theburningmonk theburningmonk.com Your code Time Extension Invoke Idle Shutdown 2s Extension shutdown Runtime shutdown Return function response Send logs
  94. 94. Distributed Tracing
  95. 95. @theburningmonk theburningmonk.com S3 Lambda DynamoDB Lambda API Gateway Lambda EventBridge Lambda Lambda
  96. 96. X-Ray
  97. 97. @theburningmonk theburningmonk.com Some services are not supported e.g. Kinesis, DynamoDB
  98. 98. @theburningmonk theburningmonk.com HTTP/HTTPs traffic only
  99. 99. @theburningmonk theburningmonk.com Requires manual instructation of AWS SDK and HTTP module
  100. 100. AWS offers a lot of built-in tools, but…
  101. 101. @theburningmonk theburningmonk.com assembly are required
  102. 102. @theburningmonk theburningmonk.com assembly are required lots of decisions for you to make
  103. 103. @theburningmonk theburningmonk.com assembly are required
  104. 104. @theburningmonk theburningmonk.com Have to use (and learn) multiple services
  105. 105. @theburningmonk theburningmonk.com Not one complete package
  106. 106. Many vendors cater for serverless workloads, but…
  107. 107. @theburningmonk theburningmonk.com Their focus is still on serverful applications, and has shallow support for serverless architectures (e.g. poor support for async workloads)
  108. 108. @theburningmonk theburningmonk.com
  109. 109. @theburningmonk theburningmonk.com github.com/theburningmonk

×