Successfully reported this slideshow.

Build reactive systems on lambda

5

Share

Loading in …3
×
1 of 152
1 of 152

Build reactive systems on lambda

5

Share

Download to read offline

Lambda is the next stage in the evolution of the AWS platform. It allows you to build reactive, event-driven systems that are easy to deploy, update and scale. Amazon manages all the undifferentiated heavy-lifting for you so you can focus on delivering value to your customers with even greater speed and cost efficiency.

Join Yan in this talk as we take a deep dive through AWS Lambda and the Serverless framework.

We'll see how to start building reactive systems using AWS Lambda, Kinesis and API Gateway, without having to manage any servers. And, you only pay for your services when they are used. We'll discuss lessons learned, best practices and current limitations with AWS Lambda.

We'll also get to know the Serverless framework, which helps automate both deployment and versioning so that you can better focus on the things that matter to your customers.

Lambda is the next stage in the evolution of the AWS platform. It allows you to build reactive, event-driven systems that are easy to deploy, update and scale. Amazon manages all the undifferentiated heavy-lifting for you so you can focus on delivering value to your customers with even greater speed and cost efficiency.

Join Yan in this talk as we take a deep dive through AWS Lambda and the Serverless framework.

We'll see how to start building reactive systems using AWS Lambda, Kinesis and API Gateway, without having to manage any servers. And, you only pay for your services when they are used. We'll discuss lessons learned, best practices and current limitations with AWS Lambda.

We'll also get to know the Serverless framework, which helps automate both deployment and versioning so that you can better focus on the things that matter to your customers.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Build reactive systems on lambda

  1. 1. + = BUILDING REACTIVE SYSTEMS WITH LAMBDA
  2. 2. hi, my name is Yan Cui @theburningmonk
  3. 3. “as a Software Engineer, what is my job?”
  4. 4. “to write code, of course!” - Me, circa 2006
  5. 5. “to provide value to users & stakeholders” - Me, 2016
  6. 6. - Dan North “lead time to someone saying thank you is the only reputation metric that matters.”
  7. 7. THE BALANCE COMPETING FORCES
  8. 8. performance security scalability simplicity cost team time to market
  9. 9. “Programmers know the benefits of everything and the tradeoffs of nothing.” - Rich Hickey https://www.infoq.com/presentations/Simple-Made-Easy
  10. 10. context is the king
  11. 11. ONLY ONE ZLATAN
  12. 12. security complexity OUTSIDE the code deployment load balancing caching monitoring config management https://www.infoq.com/presentations/complexity-simplicity-esb centralised logging elastic scaling setup server
  13. 13. Leaky Abstraction
  14. 14. AWS re:invent 2014https://www.youtube.com/watch?v=9eHoyUVo-yg
  15. 15. AWS re:invent 2014https://www.youtube.com/watch?v=9eHoyUVo-yg FAAS (Function-As-A-Service)
  16. 16. THERE IS NO SERVER
  17. 17. cost saving
  18. 18. charged by invocations
  19. 19. not paying for idle servers
  20. 20. elastic scaling
  21. 21. hi guys, come back at 10 to find out who won!
  22. 22. everyone came back at exactly 10pm!
  23. 23. everyone came back at exactly 10pm! Lambda scaled instantly
  24. 24. these spikes happen to us ALL THE TIME
  25. 25. minimise undifferentiated heavy-lifting
  26. 26. “what about containers?”
  27. 27. important, but invisible subsystems
  28. 28. economy of scale
  29. 29. NoOps!!
  30. 30. reactive
  31. 31. simple, fast deployment
  32. 32. first Lambda function went live (team size : 4)
  33. 33. (team size : 7) 155 prod deployments!
  34. 34. - Dan North “lead time to someone saying thank you is the only reputation metric that matters.”
  35. 35. easy to get started
  36. 36. now, we stand at 130 Lambda functions in prod, with 170 in dev
  37. 37. fuelling the Yubl platform evolution
  38. 38. My Beautiful Monolith
  39. 39. “What would a good architecture for us look like?”
  40. 40. small, incremental deployments fast deployments no downtime no lock-step deployments features deployed independently features loosely-coupled through messages minimise cost for unused resources minimise ops effort
  41. 41. oh, and do all these whilst continue to deliver value, thanks :-) love, your users
  42. 42. completely rebuilt search
  43. 43. Legacy Monolith Amazon Kinesis Amazon Lambda Amazon CloudSearch
  44. 44. Legacy Monolith Amazon Kinesis Amazon Lambda Amazon CloudSearchAmazon API Gateway Amazon Lambda
  45. 45. analytics pipeline
  46. 46. Legacy Monolith Amazon Kinesis Amazon Lambda Google BigQuery
  47. 47. Legacy Monolith Amazon Kinesis Amazon Lambda Google BigQuery 1 developer, 2 days design production (his 1st serverless project)
  48. 48. Legacy Monolith Amazon Kinesis Amazon Lambda Google BigQuery “nothing ever got done this fast at Skype!” - Chris Twamley
  49. 49. - Dan North “lead time to someone saying thank you is the only reputation metric that matters.”
  50. 50. Rebuilt with Lambda
  51. 51. Rebuilt with Lambda
  52. 52. and many more…
  53. 53. “JavaScript doesn’t scale well with complexity” - People on the Internet
  54. 54. but
  55. 55. “limit complexity, and JavaScript isn’t so bad” - Me
  56. 56. TESTING
  57. 57. amzn.to/29Lxuzu
  58. 58. Level of Testing 1.Unit do our objects do the right thing? are they easy to work with?
  59. 59. Level of Testing 1.Unit 2.Integration does our code work against code we can’t change?
  60. 60. handler
  61. 61. handler test by invoking the handler
  62. 62. Level of Testing 1.Unit 2.Integration 3.Acceptance does the whole system work?
  63. 63. Level of Testing unit integration acceptance
  64. 64. Level of Testing unit integration acceptance can do all 3 with Lambda
  65. 65. “…We find that tests that mock external libraries often need to be complex to get the code into the right state for the functionality we need to exercise. The mess in such tests is telling us that the design isn’t right but, instead of fixing the problem by improving the code, we have to carry the extra complexity in both code and test…” Don’t Mock Types You Can’t Change
  66. 66. “…The second risk is that we have to be sure that the behaviour we stub or mock matches what the external library will actually do… Even if we get it right once, we have to make sure that the tests remain valid when we upgrade the libraries…” Don’t Mock Types You Can’t Change
  67. 67. Don’t Mock Types You Can’t Change Services
  68. 68. “…Wherever possible, an acceptance test should exercise the system end-to- end without directly calling its internal code. An end-to-end test interacts with the system only from the outside: through its interface…” Testing End-to-End
  69. 69. Legacy Monolith Amazon Kinesis Amazon Lambda Amazon CloudSearchAmazon API Gateway Amazon Lambda
  70. 70. Legacy Monolith Amazon Kinesis Amazon Lambda Amazon CloudSearchAmazon API Gateway Amazon Lambda Test Input
  71. 71. Legacy Monolith Amazon Kinesis Amazon Lambda Amazon CloudSearchAmazon API Gateway Amazon Lambda Test Input Validate
  72. 72. “…We prefer to have the end-to-end tests exercise both the system and the process by which it’s built and deployed… This sounds like a lot of effort (it is), but has to be done anyway repeatedly during the software’s lifetime…” Testing End-to-End
  73. 73. Jenkins build config deploys and tests unit + integration tests deploy acceptance tests
  74. 74. build.sh allows repeatable builds on both local & CI
  75. 75. LOGGING
  76. 76. 2016-07-12T12:24:37.571Z 994f18f9-482b-11e6-8668-53e4eab441ae GOT is off air, what do I do now?
  77. 77. 2016-07-12T12:24:37.571Z 994f18f9-482b-11e6-8668-53e4eab441ae GOT is off air, what do I do now? UTC Timestamp API Gateway Request Id your log message
  78. 78. organised by Function + Version
  79. 79. LOG OVERLOAD
  80. 80. centralise your logs
  81. 81. CloudWatch Logs AWS Lambda LogStash ElasticSearch
  82. 82. CloudWatch Logs AWS Lambda LogStash ElasticSearch AWS Elasticsearch
  83. 83. CloudWatch Logs AWS Lambda LogStash ElasticSearch AWS Elasticsearch Elastic Cloud
  84. 84. CloudWatch Logs AWS Lambda LogStash ElasticSearch AWS Elasticsearch Elastic Cloud ?
  85. 85. correlation IDs
  86. 86. MONITORING
  87. 87. nowhere to install monitoring agents
  88. 88. no time for background tasks
  89. 89. • Invocation Count • Error Count • Latency (avg, min, max, sum) • Throttling • Granular to the minute • Support custom metrics
  90. 90. • Same basic metrics • Better dashboard • Support custom metrics https://www.datadoghq.com/blog/monitoring-lambda-functions-datadog/
  91. 91. Do It Yourself
  92. 92. PRO TIP set up dashboards
  93. 93. PRO TIP don’t forget to set up alarms
  94. 94. PRO TIP add application-level metrics
  95. 95. KEEP WARM
  96. 96. functions are unloaded if idle for a while
  97. 97. noticeable cold start time (package size matters)
  98. 98. CloudWatch Event AWS Lambda
  99. 99. CloudWatch Event AWS Lambda ping ping ping ping
  100. 100. CloudWatch Event AWS Lambda ping ping ping ping
  101. 101. CloudWatch Event AWS Lambda ping ping ping ping HEALTH CHECKS?
  102. 102. KNOW YOUR LIMITS
  103. 103. max 50 MB deployment package size
  104. 104. max 50 MB deployment package size max 75 GB total deployment package size* * limit is per AWS region
  105. 105. Janitor Monkey
  106. 106. Janitor Lambda
  107. 107. max 5 mins execution time
  108. 108. max 6 MB request payload size* max 6 MB response payload size * for a request-response event type
  109. 109. default max 100 concurrent executions* * soft-limit, can be raised via support ticket
  110. 110. looking ahead
  111. 111. .Net core? SQS support?
  112. 112. v1.0
  113. 113. MULTI-CLOUD FUTURE?
  114. 114. IBM OpenWhisk Amazon Lambda Azure Web Functions Google Cloud Functions competition faster innovation lower prices
  115. 115. @theburningmonk @theburningmonk theburningmonk.com github.com/theburningmonk
  116. 116. @theburningmonk @theburningmonk theburningmonk.com github.com/theburningmonk slides at http://bit.ly/2ecqEql

×