Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Essential open source tools for serverless developers

1,497 views

Published on

In this session, AWS Serverless Hero Yan Cui will share the open-source tools and frameworks that he has found so useful in his journey with serverless. The list features libraries, deployment frameworks, plugins, middlewares, Lambda layers and much more!

Published in: Technology
  • Be the first to comment

Essential open source tools for serverless developers

  1. 1. ESSENTIAL OPEN SOURCE TOOLS FOR THE #SERVERLESS DEVELOPER
  2. 2. DEPLOYMENT FRAMEWORKS
  3. 3. @theburningmonk theburningmonk.com
  4. 4. @theburningmonk theburningmonk.com https://lumigo.io/blog/comparison-of-lambda-deployment-frameworks
  5. 5. @theburningmonk theburningmonk.com don’t write your own deployment framework
  6. 6. @theburningmonk theburningmonk.com
  7. 7. @theburningmonk theburningmonk.com no extensibility
  8. 8. @theburningmonk theburningmonk.com great for infratructure (VPCs, networking, etc.) terribly unproductive for constructing a serverless application
  9. 9. @theburningmonk theburningmonk.com
  10. 10. DON’T automate what you don’t understandDON’T automate what you don’t understand
  11. 11. @theburningmonk theburningmonk.com beware of vanity features
  12. 12. @theburningmonk theburningmonk.com A brogrammer is a male programmer with traits often associated with fraternity brothers (bros) … Brogrammers
  13. 13. @theburningmonk theburningmonk.com “Python is not real code. Real programmers use Java.”
  14. 14. @theburningmonk theburningmonk.com “…the c in iac is stand for code while in Serverless it’s stand for configuration because the serverless.yml is no really a code”
  15. 15. @theburningmonk theburningmonk.com qualities you should care about IAC
  16. 16. @theburningmonk theburningmonk.com qualities you should care about IAC syntactic correctness implies semantic correctness only one way to interpret things
  17. 17. @theburningmonk theburningmonk.com qualities you should care about IAC syntactic correctness implies semantic correctness only one way to interpret things concise
  18. 18. @theburningmonk theburningmonk.com qualities you should care about IAC syntactic correctness implies semantic correctness only one way to interpret things concise interoperable
  19. 19. @theburningmonk theburningmonk.com https://forrestbrazeal.com/2020/01/05/code-wise-cloud-foolish-avoiding-bad-technology-choices
  20. 20. @theburningmonk theburningmonk.com qualities you should care about IAC syntactic correctness implies semantic correctness only one way to interpret things concise interoperable
  21. 21. @theburningmonk theburningmonk.com https://acloud.guru/series/serverlessconf-nyc-2019/view/yaml-better
  22. 22. Picking a deployment framework because it lets you write Javascript is like picking a car because it has red seats. Your priorities are WRONG!
  23. 23. Yan Cui http://theburningmonk.com @theburningmonk AWS user for 10 years
  24. 24. http://bit.ly/yubl-serverless
  25. 25. Yan Cui http://theburningmonk.com @theburningmonk Developer Advocate @
  26. 26. Yan Cui http://theburningmonk.com @theburningmonk Independent Consultant advisetraining delivery
  27. 27. https://theburningmonk.com/courses
  28. 28. @theburningmonk theburningmonk.com https://serverless.com/plugins
  29. 29. @theburningmonk theburningmonk.com https://github.com/functionalone/serverless-iam-roles-per-function
  30. 30. @theburningmonk theburningmonk.com
  31. 31. @theburningmonk theburningmonk.com https://github.com/serverless-heaven/serverless-webpack
  32. 32. @theburningmonk theburningmonk.com https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk
  33. 33. @theburningmonk theburningmonk.com https://github.com/dherault/serverless-offline
  34. 34. @theburningmonk theburningmonk.com
  35. 35. @theburningmonk theburningmonk.com https://github.com/amplify-education/serverless-domain-manager
  36. 36. @theburningmonk theburningmonk.com
  37. 37. @theburningmonk theburningmonk.com https://github.com/horike37/serverless-step-functions
  38. 38. @theburningmonk theburningmonk.com
  39. 39. @theburningmonk theburningmonk.com API Gateway trigger Events trigger Schedule trigger
  40. 40. @theburningmonk theburningmonk.com CloudWatch Alarms CloudWatch Notifications
  41. 41. @theburningmonk theburningmonk.com Validation Blug-green deployment Express Workflows
  42. 42. @theburningmonk theburningmonk.com https://github.com/fernando-mc/serverless-finch
  43. 43. @theburningmonk theburningmonk.com
  44. 44. CLIs
  45. 45. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  46. 46. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  47. 47. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  48. 48. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  49. 49. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  50. 50. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  51. 51. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  52. 52. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  53. 53. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation
  54. 54. @theburningmonk theburningmonk.com https://github.com/OlafConijn/AwsOrganizationFormation Accounts Org Units SCPs Pwd Policies Multi-Region Pseudo-Funs Init & Validate CI/CD
  55. 55. @theburningmonk theburningmonk.com https://github.com/lumigo-io/lumigo-cli
  56. 56. @theburningmonk theburningmonk.com list lambda functions
  57. 57. @theburningmonk theburningmonk.com analyze lambda costs
  58. 58. @theburningmonk theburningmonk.com analyze lambda cold starts
  59. 59. @theburningmonk theburningmonk.com tune lambda memory setting
  60. 60. @theburningmonk theburningmonk.com tune lambda memory setting
  61. 61. @theburningmonk theburningmonk.com tail DynamoDB stream
  62. 62. @theburningmonk theburningmonk.com tail SQS queue
  63. 63. @theburningmonk theburningmonk.com tail SNS topic
  64. 64. @theburningmonk theburningmonk.com tail EventBridge bus & rule
  65. 65. @theburningmonk theburningmonk.com replay SQS DLQ events
  66. 66. @theburningmonk theburningmonk.com whoami & switch-profile
  67. 67. @theburningmonk theburningmonk.com
  68. 68. Libraries
  69. 69. @theburningmonk theburningmonk.com https://github.com/lambci/docker-lambda
  70. 70. @theburningmonk theburningmonk.com
  71. 71. @theburningmonk theburningmonk.com https://github.com/middyjs/middy
  72. 72. @theburningmonk theburningmonk.com
  73. 73. @theburningmonk theburningmonk.com
  74. 74. @theburningmonk theburningmonk.com
  75. 75. @theburningmonk theburningmonk.com secrets should NEVER be in plain text in env variables
  76. 76. @theburningmonk theburningmonk.com SSM Parameter Store Secret 1 Secret 2 IAM Environment: SECRET_1: … SECRET_2: … Environment: SECRET_1: … SECRET_2: …
  77. 77. @theburningmonk theburningmonk.com SSM Parameter Store Secret 1 Secret 2 IAM Environment: SECRET_1: … SECRET_2: … Environment: SECRET_1: … SECRET_2: … yay!
  78. 78. @theburningmonk theburningmonk.com
  79. 79. @theburningmonk theburningmonk.com
  80. 80. @theburningmonk theburningmonk.com SSM Parameter Store Secret 1 Secret 2 IAM fetch at cold start, cache, invalidate every x mins
  81. 81. @theburningmonk theburningmonk.com
  82. 82. @theburningmonk theburningmonk.com
  83. 83. @theburningmonk theburningmonk.com SSM Parameter Store Secret 1 Secret 2 IAM switch to Higher Throughput if you need more than 40 ops/s
  84. 84. @theburningmonk theburningmonk.com https://github.com/getndazn/dazn-lambda-powertools
  85. 85. @theburningmonk theburningmonk.com
  86. 86. @theburningmonk theburningmonk.com
  87. 87. @theburningmonk theburningmonk.com use middleware to auto-capture incoming correlation IDs
  88. 88. @theburningmonk theburningmonk.com
  89. 89. extract correlation IDs from invocation event, and store them in the correlation-ids module reset
  90. 90. @theburningmonk theburningmonk.com
  91. 91. @theburningmonk theburningmonk.com intercepts and extracts correlation IDs
  92. 92. @theburningmonk theburningmonk.com automatically include extracted correlation IDs
  93. 93. @theburningmonk theburningmonk.com
  94. 94. @theburningmonk theburningmonk.com HTTP and AWS SDK clients to auto-forward correlation IDs on
  95. 95. @theburningmonk theburningmonk.com context.awsRequestId get-index
  96. 96. @theburningmonk theburningmonk.com context.awsRequestId x-correlation-id get-index
  97. 97. @theburningmonk theburningmonk.com { “headers”: { “x-correlation-id”: “…” }, … } get-index
  98. 98. @theburningmonk theburningmonk.com { “body”: null, “resource”: “/restaurants”, “headers”: { “x-correlation-id”: “…” }, … } get-index get-restaurants
  99. 99. @theburningmonk theburningmonk.com get-restaurants global.CONTEXT global.CONTEXT x-correlation-id = … x-correlation-xxx = … get-index headers[“User-Agent”] headers[“Debug-Log-Enabled”] headers[“User-Agent”] headers[“Debug-Log-Enabled”] headers[“x-correlation-id”] capture forward function event log.info(…)
  100. 100. @theburningmonk theburningmonk.com CloudWatch Logs $0.50 per GB ingested $0.03 per GB archived per month
  101. 101. @theburningmonk theburningmonk.com CloudWatch Logs $0.50 per GB ingested $0.03 per GB archived per month 1M invocation of a 128MB function = $0.000000208 * 1M + $0.20 = $0.408
  102. 102. @theburningmonk theburningmonk.com DON’T leave debug logging ON in production
  103. 103. @theburningmonk theburningmonk.com
  104. 104. @theburningmonk theburningmonk.com have to redeploy ALL the functions along the call path to collect all relevant debug logs
  105. 105. @theburningmonk theburningmonk.com
  106. 106. @theburningmonk theburningmonk.com
  107. 107. @theburningmonk theburningmonk.com
  108. 108. @theburningmonk theburningmonk.com
  109. 109. @theburningmonk theburningmonk.com
  110. 110. @theburningmonk theburningmonk.com sampling decision is forwarded on and followed by the entire call chain
  111. 111. SARServerless Application Repository
  112. 112. @theburningmonk theburningmonk.com https://go.aws/2t3rQ7a
  113. 113. @theburningmonk theburningmonk.com https://go.aws/2NZ8dUW
  114. 114. @theburningmonk theburningmonk.com tune lambda memory setting
  115. 115. @theburningmonk theburningmonk.com https://go.aws/38HAqYe
  116. 116. @theburningmonk theburningmonk.com https://go.aws/2tyBjDH
  117. 117. https://go.aws/2TAQhno @theburningmonk theburningmonk.com
  118. 118. @theburningmonk theburningmonk.com https://go.aws/38KynD1
  119. 119. @theburningmonk theburningmonk.com serverless-iam-roles-per-function serverless-webpack serverless-offline serverless-domain-manager serverless-step-functions serverless-finch org-formation lumigo-cli docker-lambda middy dazn-lambda-powertools lambda-janitor aws-lambda-power-tuning s3-lambda-publisher-sns auto-subscribe-log-group-to-arn auto-set-log-group-retention sfn-callback-urls
  120. 120. https://theburningmonk.com/hire-me AdviseTraining Delivery “Fundamentally, Yan has improved our team by increasing our ability to derive value from AWS and Lambda in particular.” Nick Blair Tech Lead
  121. 121. @theburningmonk theburningmonk.com https://theburningmonk.com/workshops Amsterdam, March 19-20 Helsinki, May 4-5 Stockholm, May 14-15 Dublin, June 16-17 London, September 24-25 Berlin, October 8-9
  122. 122. @theburningmonk theburningmonk.com Production-Ready Serverless
  123. 123. @theburningmonk theburningmonk.com bit.ly/prod-ready-sls-stockholm-2020 20% off with awscommday-0131

×