The Good, the Bad and the Ugly of Serverless Computing
•Download as PPTX, PDF•
0 likes•1,415 views
Martin Raag, Head of Engineering at Testlio Real world experience operating a serverless architecture
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to The Good, the Bad and the Ugly of Serverless Computing
Similar to The Good, the Bad and the Ugly of Serverless Computing (20)
Recently uploaded
Recently uploaded (20)
The Good, the Bad and the Ugly of Serverless Computing
- 1. Serverless The Good, the Bad and the Ugly Martin Raag martin@testlio.com
- 2. Functions as a Service
- 3. Lambda DynamoDB S3 Kinesis Lambda CloudFormation SWF CloudTrail SES Cognito API Gateway CloudWatchSNS
- 5. w
- 9. The serverless manifesto • No machines, VMs, or containers visible in the programming model. • Functions are the unit of deployment & scaling. • Implicitly fault tolerant, as functions can run anywhere. • Scales per request. • Users can not over- or under-provision capacity. • Permanent storage lives elsewhere. • Never pay for idle, no cold servers or their costs. • Bring your own code (BYOC). • Metrics and logging are a universal right.
- 11. Latency
- 12. Latency • Coldstarts add ~100ms latency • Up to ~10s for JVM based functions • Keep your functions warm
- 13. Latency functions: hello: handler: handler.hello events: - http: path: hello method: get - schedule: rate(10 minutes)
- 18. API Gateway
- 19. – Top Gear “Ambitious, but rubbish.”
- 20. What API Gateway does? • Authentication • Caching • Input validation • Request and response mapping • Throttling • Stages • Environment variables
- 21. What API Gateway doesn’t? • Integrate (well) with other AWS services
- 22. API Gateway • No CloudFront edge cache • Lambda timeout at 10s (maybe 30s) • No Certificate Manager integration
- 26. Configuration
- 28. – Adam Wiggins (12 Factor App) “The twelve-factor app stores config in environment variables.”
- 31. Debugging
- 33. DoS yourself
- 35. That’s not all • No persistent state • 5 minute execution limit • No atomic deployments of a service • Immature tooling • Vendor control / lock-in • Service Discovery
- 36. Conclusion
- 38. Release Your App With Confidence Martin Raag martin@testlio.com
Editor's Notes
- Testlio is a full-service app testing solution, that allows you to offload your QA with the most flexible requirements. All we need is your build and you will have a dedicated team of professional software testers.
- A serverless application is one where the server-side logic runs in stateless compute containers that are event-triggered, ephemeral (may last only a single execution) and fully managed by a 3rd party. Essentially you can think about it as Functions as a Service (FaaS). There are multiple service providers today - Googles Cloud Functions, Azures Service Fabric and Auth0’s webtasks, but the first and perhaps the most popular implementation today is AWS’ Lambda.
- First introduced in November of 2014, Lambda integrated with AWS’ DynamoDB (a managed key value store), S3 (cloud file storage) and Kinesis (data processing) as event sources. A year later AWS expanded support across their services (CloudFormation, SWF, CloudTrail, SES, Cognito, API Gateway, SNS, CloudWatch). Lets focus on some of these to see how they enable us to build serverless cloud services.
- API Gateway - Allows us to map individual Lambda functions to HTTP endpoints to create Restful services. DynamoDB - A fully managed database service to enable persistent data storage and lookup without the overhead of database maintenance. CloudWatch - Centralised logging for all Lambda functions and periodic triggers. CloudFormation - Enables to define and deploy a complete sandboxed stack.
- Updating servers (OS’), Deploying code to a cluster, Horizontal scaling is completely automatic, pay for what you use (subsecond billing w/ 100ms accuracy)
- You might be hyped. But before you jump into writing everything using the serverless paradigm, lets look at some of the bad problems and ugly workarounds.
- In general your serverless service will be slower.
- • Cold boots add about ~100ms depending on the size of your function and runtime (Node, Python) • JVM functions may take significantly longer (up to ~10s) • The ugly workaround is to ping your endpoint periodically or set a periodic event
- Actually pretty easy with serverless framework, which abstracts some of the complexity away - just one configuration line. A schedule does not completely solve the problem though.
- Even if your function is under constant load the VM seems to be recycled every ~4 hours. IOPipe data - AWS does not say. Coldstarts are not the only problem.
- Latency in the same region (us-east-1) is similar. The traditional approach comes in just under ~100ms, while the serverless is just over it. Both pretty consistent. When we look at cross region requests things get a little funky though.
- Lambda/APIGW approach is at times much slower and also a lot more inconsistent. APIGW is based on CloudFront, you reach the Amazon network quickly, but the Amazon network seems at times more unreliable than the internet.
- time_connect - from the start until the TCP connect to the remote host APIGW/Lambda very fast - latency seems to happen in the Amazon network.
- API Gateway - I have a love - hate relationship with that Amazon Service. At times it seems to follow the old Top Gear moto - “Ambitious, but rubbish”.
- If you look at the service offering, it tries to do so much. Some of these things are better remained in application code, as they are hard to debug, version control and sometimes even define.
- Cache does not help with Amazon network problems, since it’s not implemented on the edge. API Gateway might timeout on coldstart. No certificate manager integration, works with ELB and CloudFront.
- What you do in this case is crawl the AWS forums for help…
- Where you will be assured it’s a high priority item in the backlog.
- Even though it probably isn’t.
- Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.
- The twelve-factor app stores config in environment variables. 1. Bake the configuration into the package (either as a file or in code) 2. Use S3 3. Use DynamoDB The first one is a bad idea for reasons which are out of the scope of this talk (see 12 Factor App), to mention a few: • you will not be able to use your build artifacts across deployments to different stages, if the configuration between them varies at all • redeploy to update configuration
- S3 and Dynamo increase latency even more. You can load configuration only once on cold start, but then you don’t know when your config change propagates.
- No general FaaS abstraction Local environment lots of manual work Containers exist for AWS services All event sources are possible to emulate, but impractical Resort to development in the cloud
- Rely on what your service provider gives you. Meaning there is no SSH access to containers or remote debugging. Debugging happens by investigating metrics and lots and lots of logging.
- By default, AWS Lambda limits the total concurrent executions across all functions within a given region to 100. The default limit is a safety limit that protects you from costs due to potential runaway or recursive functions during initial development and testing.
- This means otherwise self contained services can impact each other. If different environments (dev, test, prod) run under the same account, then those can also affect each other. Load testing dev/test env Processing big data Results in unresponsive API
- database connection pools / persistent HTTP connections to external services are also persistent state recursive or spawn multiple lambdas to avoid execution time limit (can lead to fun stuff with the concurrent execution limit) might have to disable event triggers to deploy a service due to no atomic deploys of a service
- Still immature Be aware of the limits Sweet spot is currently along with servers Very powerful and will become more prominent