Running serverless at scale

1. running serverless at SC AL E

2. Yan Cui http://theburningmonk.com @theburningmonk Principal Engineer @ Independent Consultant

5. available in Austria, Switzerland, Germany, Japan, Canada, Italy, US, Spain and Brazil

6. available on 30+ platforms

7. ~1,000,000 concurrent viewers

8. follow @dazneng for updates about the engineering team We’re hiring! Visit engineering.dazn.com to learn more. WE’RE HIRING!

9. AWS user since 2009

10. AWS user since 2009

11. What do you mean by ‘serverless’?

12. “Serverless”

14. Gojko Adzic It is serverless the same way WiFi is wireless. http://bit.ly/2yQgwwb

15. Serverless means… don’t pay for it if no-one uses it don’t need to worry about scaling don’t need to provision and manage servers

16. “Function-as-a-Service” AWS Lambda Azure Functions Google Cloud Functions Auth0 Webtask Spotinst Functions Kubeless IBM Cloud Functions

17. AWS Lambda

18. AWS Lambda API Gateway IOT SNS Kinesis CloudWatch

19. IaaS Function Application Runtime Container OS Virtualization Hardware CaaS Function Application Runtime Container OS Virtualization Hardware PaaS Function Application Runtime Container OS Virtualization Hardware FaaS Function Application Runtime Container OS Virtualization Hardware User User (scalable unit) Provider

20. IaaS Function Application Runtime Container OS Virtualization Hardware CaaS Function Application Runtime Container OS Virtualization Hardware PaaS Function Application Runtime Container OS Virtualization Hardware FaaS Function Application Runtime Container OS Virtualization Hardware User User (scalable unit) Provider

21. Serverless FaaS other services… Database Storage BI

22. Simon Wardley Serverless will fundamentally change how we build business around technology and how you code.

23. Why serverless?

24. more Scalable

25. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)

26. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit) AUTO-APPROVED RAISE TO 3000

28. containers are reused

29. 100% SERVERLESS IN PRODUCTION

30. 80 MILLION MONTHLY USERS

31. Cheaper (don’t pay for idle servers)

32. Resilience (built-in redundancy and multi-AZ)

33. http://bit.ly/2Vzfexo

34. Secure

35. Shared Responsibility Model

36. Shared Responsibility Model

37. protection from OS attacks Amazon automatically apply latest patches to host VMs

40. Deploy

41. serverless.yml {} Code

42. {} Code serverless.yml

43. serverless.yml {} Code S3

44. {} Code serverless.yml S3 CloudFormation

45. {} Code serverless.yml S3 CloudFormation

46. request blue-green deployment

49. request blue-green deployment req/s auto-scaling us-east-1a us-east-1b us-east-1c multi-AZ

50. the DevOps forcethe DevOps force is strong with serverlessis strong with serverless

51. idea production choose language + framework master language + framework figure out deployment configure AMI configure ELB configure autoscaling capacity planning over-provision for launch are we doing microservices? configure CI/CD

52. idea production choose language + framework master language + framework figure out deployment configure AMI configure ELB configure autoscaling capacity planning over-provision for launch are we doing microservices? configure CI/CD

53. idea production greater Velocity from idea to product

54. minimise undifferentiated heavy-lifting

55. less ops responsibility on your shoulders

56. infrastructure you

61. ~1,000,000 concurrent viewers

63. serverless is not right for every use case (yet)

64. http://bit.ly/2WSfcky

65. Lambda VPC

66. there are no silver bullets

68. 0 Containers “it works on my machine!” “production ready!”days

69. 0 Containers “it works on my machine!” “production ready!”days Serverless 0 “it works!” “production ready!” days

70. 0 Containers “it works on my machine!” “production ready!”days Serverless 0 “it works!” “production ready!” days v2! v3! v4! v5! v6!

71. EC2 docker us-east-1a us-east-1b us-east-1a us-east-1b Theory

72. Reality

73. Reality

74. scale-to-zero

75. serverful serverless us-east-1a us-east-1b us-east-1a us-east-1bscaled to zero!

76. scaling limits VPC long-running cold starts performance

79. http://bit.ly/2X0ksCY

83. http://bit.ly/2I7GJeJ

87. What can we do now?

92. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit) contact your AWS TAM about raising this

93. multi-region, active-active

95. overall trafﬁc regional trafﬁcs

96. http://bit.ly/2Vzfexo

98. identical, independent

100. ingest from both regions

101. ingest from both regions idempotent

108. CloudFront API Gateway Lambda

109. cache as close to the user as possible!

110. CloudFront API Gateway Lambda cache

111. CloudFront API Gateway Lambda cache cache

112. CloudFront API Gateway Lambda cache cache cache take advantage of container reuse

113. https://amzn.to/2VlyXzR

114. CloudFront Lambda@Edge executed in the nearest AWS region

115. CloudFront Lambda@Edge executed in the nearest AWS region trafﬁc is distributed across many regions

116. CloudFront Lambda@Edge executed in the nearest AWS region trafﬁc is distributed across many regions fewer hops, shorter round- trip = faster response

117. avoid services that require persistent connections

118. DynamoDB on-demand tables can scale to up to 40K TPS DynamoDB

119. RDS requires persistent connection

120. RDS ๏ set client-side pool size to 1 ๏ set server-side max connection high ๏ set server-side connection timeout to ~10 mins (to clean phantom connections) ๏ use read-replicas

121. RDS ๏ set client-side pool size to 1 ๏ set server-side max connection high ๏ set server-side connection timeout to ~10 mins (to clean phantom connections) ๏ use read-replicas Serverless Aurora is not ready for production-use (yet)

122. RDS new service HTTPs socket pooling, etc.

124. requires persistent connection

126. https://github.com/solve-hq/LaunchDarkly-relay-fargate

127. function-less

128. CloudFront API Gateway Lambda 500/min limit on scaling out

129. CloudFront API Gateway Lambda

130. CloudFront API Gateway DynamoDB

131. CloudFront AppSync DynamoDB

132. engineer the problem away instead of solving it high 5!

133. What about cold starts?

134. INITIALISE CONTAINER INITIALISE RUNTIME INITIALISE HANDLER YOUR CODE EXECUTES

135. INITIALISE CONTAINER INITIALISE RUNTIME INITIALISE HANDLER YOUR CODE EXECUTES COLD START

136. “cold starts only happen to the ﬁrst request”

137. “cold starts only happen to the ﬁrst request” WRONG!!!

138. function invocationconcurrent execution i.e. a container

139. function invocationconcurrent execution i.e. a container class instance method call

140. Lambda scales the number of concurrent executions based on trafﬁc

141. existing “containers” are reused where possible

142. time invocation

143. time invocation invocation

144. time invocation invocation

145. time invocation invocation invocation invocation

146. time invocation invocation invocation invocation invocation invocation

147. time invocation invocation invocation invocation invocation invocation

148. time invocation invocation invocation invocation invocation invocation invocation

149. time invocation invocation invocation invocation invocation invocation invocation invocation

156. FREQUENCY DURATION

157. FREQUENCY DURATION dictated by user trafﬁc, out of your control

158. FREQUENCY DURATION optimize this!

159. “Lambda is not a suitable solution for me because of cold starts”

160. “what is your latency requirement?”

161. cold starts that don’t add to user-facing latency is generally not worth worrying about

162. Node.js functions, no VPC, 1GB, averages ~500ms cold start with production workload

163. Node.js functions, no VPC, 1GB, averages ~500ms cold start with production workload (good enough for most web applications)

164. sporadic spikes latency existed before Lambda

165. GC pauses…

166. overloaded servers…

167. slow downstream, databases, etc.

168. networking issues…

169. cold starts is generally not an issue if you have a steady trafﬁc pattern

170. time req/s

171. time req/s El Classico

172. time req/s lunch dinner

173. minimise the duration of cold starts so they fall within acceptable latency range

174. use Node.js, Python or Golang

175. trim dependencies

176. don’t require the full AWS-SDK if you only need one client

177. https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk/

178. https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk/ full AWS-SDK vs. DynamoDB only

179. https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk/ full AWS-SDK vs. DynamoDB only webpack!!

180. keep functions single-purposed

181. avoid VPCs unless you need to access VPC-protected resources

183. caching, caching, caching,

184. multi-region, active-active

185. horizontal scalability all the way

186. lambda warmer doesn’t work

187. make cold start durations acceptable

188. “not everything at Netflix runs at Netflix scale”

189. @theburningmonk theburningmonk.com github.com/theburningmonk

Running serverless at scale

Running serverless at scale

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Running serverless at scale

Similar to Running serverless at scale(20)

More from Yan Cui

More from Yan Cui(20)

Recently uploaded

Recently uploaded(20)

Running serverless at scale