Matt Williams, Datadog Just as we got a hang of monitoring our server-based applications, they take away the server. How do you monitor something that doesn’t exist? What metrics matter most in a serverless world? In this session, we will look at how applications are different in a AWS Lambda-based world and how to monitor them. Join us as we work our way through the stack and demonstrate how to capture the health and performance of your services. The focus of this session is not tool specific. Attendees will learn production tested lessons and leave with frameworks they can implement with their serverless workloads regardless of the platforms and tools they use.

1. November 15, 2016
Monitoring, Hold the Infrastructure
Getting the most from AWS Lambda
Matt Williams – Evangelist @ Datadog
@technovangelist - mattw@datadoghq.com

2. November 15, 2016
The Plan
Brief Datadog Overview
The what and the why of Lambda
Adding Lambda to your infrastructure
Monitoring 101 (tl;dr version)
Adding Lambda to your monitoring

3. November 15, 2016
Datadog Overview
SaaS-based Infrastructure Monitoring
Focus on Modern Infrastructure
(cloud, containers, micro-services)
Processing about a trillion metrics per day
Intelligent alerting

4. November 15, 2016
Monitor Everything

5. November 15, 2016
The Plan
Brief Datadog Overview
The what and the why of Lambda
Adding Lambda to your infrastructure
Monitoring 101 (tl;dr version)
Adding Lambda to your monitoring

6. November 15, 2016
What is Lambda
The next stage in a long progression...
physical
boxes
virtual
machines
on
physical
boxes
containers lambda
AWS
instances

7. November 15, 2016
Important Lambda Concept

8. November 15, 2016
Important Lambda Concepts

9. November 15, 2016
Important Lambda Concepts
SERVERLESS
AS FAR AS YOU ARE CONCERNED

10. November 15, 2016
What is a AWS Lambda Function
Each AWS Lambda Function is:
• Code in a container on LXC on AMZN Linux
– Container implementation hidden
• No restrictions on what it can do
– Runtimes for Java, Node, Python
– Other languages via exec/spawn
• Priced based on exec time, quantity, and memory

11. November 15, 2016
What can trigger AWS Lambda?
Various AWS Services including:
Files (S3)
Database Updates (DynamoDB)
Streams (Kinesis)
Messages (SNS, SES)
IoT (Echo, API Gateway)

12. November 15, 2016
So What Are Some Good Use Cases
Bootstrapping a startup (first 1M requests/month free, up to 3.2M free
seconds / month)
Cron
File manipulation (transcode, compress, etc)
API translation
Stream processing (social media analysis)
Scaling infrastructure
Mobile/IoT Backends
Alexa Skills

13. November 15, 2016
https://goad.io/

14. November 15, 2016
Other Examples
Bittorrent Tracker
• Hugely scalable
• Uses S3 for assets
• Capable of handling 9,000,000 peers
https://blog.zappa.io/posts/zappa-bittorrent-tracker
Chat Bots
• Easy to build
https://claudiajs.com/claudia-bot-builder.html
S3 Data Loading
• Simpler than the typical Kinesis process
• (but slower)
https://www.trek10.com/blog/serverless-
architectures-s3-data-loading/

15. November 15, 2016
The Plan
Datadog overview
The what and the why of AWS Lambda
Adding Lambda to your infrastructure
Monitoring 101 (the tl;dr version)
Adding AWS Lambda to your monitoring

16. November 15, 2016
How to Create a Lambda Function
1. Write the code
2. Setup on AWS (mem, timeout, role, handler)
– Web interface
– node-lambda, apex, serverless, etc
3. Debug
– logs

17. November 15, 2016
AWS Lambda Demo

18. November 15, 2016
Using node-lambda
• npm install node-lambda
• npm run setup
• npm run dry-run
• npm run deploy

19. November 15, 2016
Using apex
• apex init
• apex deploy
• apex deploy --dry-run
• apex invoke

20. November 15, 2016
Using serverless
• npm install serverless –g
• serverless create --template
aws-nodejs
• serverless deploy
• serverless invoke

21. November 15, 2016
Native Gotchas
• Remember that Lambda runs on an Amazon
Linux container
• …and your dev box probably isn’t Amazon Linux…
• …so native NPM modules are problematic

22. November 15, 2016
Native npm
1. Launch an Amazon Linux ec2
2. Install nvm (https://github.com/creationix/nvm)
3.nvm install 4.3
4.npm install <chosen modules>
5. Zip up node_modules
6. scp to dev box, unzip
7. Deploy

23. November 15, 2016
Other AWS Lambda Tools
• Apex (http://apex.run)
• Serverless framework (http://serverless.com)
• GoSparta (http://gosparta.io)
• go-lambda (https://github.com/xlab/go-lambda)
• AWS Chalice
(https://github.com/awslabs/chalice)

24. November 15, 2016
The Plan
Datadog overview
The what and the why of AWS Lambda
Adding AWS Lambda to your infrastructure
Monitoring 101 (the tl;dr version)
Adding AWS Lambda to your monitoring

25. November 15, 2016

26. November 15, 2016

27. November 15, 2016
Collecting data is cheap

28. November 15, 2016
Collecting data is cheap;
Not having it when you
need it can be expensive

29. November 15, 2016

30. November 15, 2016

31. November 15, 2016

32. November 15, 2016

33. November 15, 2016

34. November 15, 2016

35. November 15, 2016

36. November 15, 2016
The Plan
Datadog overview
The what and the why of AWS Lambda
Adding AWS Lambda to your infrastructure
Monitoring 101 (the tl;dr version)
Adding Lambda to your monitoring

37. November 15, 2016
What are the AWS Lambda Metrics
aws.lambda.duration, min, max, sum
aws.lambda.errors
aws.lambda.invocations
aws.lambda.throttles

38. November 15, 2016
3 Options for Monitoring Lambda
1. Add a custom metric to CloudWatch
2. Add metric directly to monitoring application
3. Add a line to the CloudWatch logs

39. November 15, 2016
The Downsides of the First Two
• CloudWatch Metrics
• CloudWatch metrics are updated every minute
• Send more than one per minute and you get a
histogram
• Directly to Monitoring App
• Adds complexity and size to your lambda (should
be kept small)
• But no lag time

40. November 15, 2016
How Datadog Does It
Add line to CloudWatch logs:
MONITORING|unix_epoch_timestamp|value|
type|my.metric.name|#tag1:value,tag2
metric type can be count or gauge
In Python: print(monitoringstring)
In NodeJS: console.log(monitoringstring)

41. November 15, 2016
Monitoring example

42. November 15, 2016
Monitor All the Lambda Things
Monitor the triggers
Monitor the function
Monitor the output

43. November 15, 2016

44. November 15, 2016
Summary
AWS Lambda can be an interesting part of your
infrastructure
Very low cost to start
Serverless doesn’t mean monitorless
Matt Williams - matt.williams@datadoghq.com
@technovangelist

45. November 15, 2016

46. November 15, 2016