Serverless computing - Build and run applications without thinking about servers

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Prakash Palanisamy
Solutions Architect, Amazon Web Services
Joachim den Hertog
Solutions Architect, ReSnap (Albelli)
Serverless computing: Build and run
applications without thinking about
servers

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Means …
No servers to provision
or manage
Scales with usage
Never pay for idle Built-in availability and
fault tolerance

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Applications
Services (anything)
Changes in
data state
Requests to
endpoints
Changes in
resource state
Event source Lambda function
Node.js
Python
Java
C#
Go

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Services for Building Serverless Applications
AWS Lambda
Amazon DynamoDB
Amazon SNS
Amazon API Gateway
Amazon SQS
Amazon Kinesis
Amazon S3
Compute and API Proxy
AWS X-Ray
AWS Step Functions
Datastores, Storage, Orchestration, Analytics, Interprocess Messaging
Developer Tools
AWS CodeBuild
AWS CodePipeline
AWS Serverless
Application Model (SAM)
Open Source and
third parties
Lambda@Edge
AWS Cloud9
Amazon Aurora
Serverless (preview)
AWS AppSync

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Common Serverless Application Use Cases
Web
applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Data
processing
• Real time
• MapReduce
• Batch
• Machine
learning
inference
Chatbots
• Powering
chatbot logic
Backends
• Apps and
services
• Mobile
• IoT
</></>
Amazon
Alexa
• Powering
voice-enabled
apps
• Alexa
Skills Kit
IT
automation
• Policy engines
• Extending
AWS services
• Infrastructure
management

6. Deep Dive/Components

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Using AWS Lambda
Bring your own code
• Node.js, Java, Python,
C#, Go
• Bring your own libraries
(even native ones)
Simple resource model
• Select power rating
from 128 MB to 3 GB
• CPU and network
allocated
proportionately
Flexible use
• Synchronous or
asynchronous
• Integrated with other
AWS services
Flexible authorization
• Securely grant access to
resources and VPCs
• Fine-grained control for
invoking your functions

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon S3 Amazon
DynamoDB
Amazon
Kinesis
AWS
CloudFormation
AWS CloudTrail Amazon
CloudWatch
Amazon
Cognito
Amazon SNSAmazon
SES
Cron events
Data stores Endpoints
Configuration repositories Event/message services
Example Event Sources That Trigger AWS Lambda
… and more on the way.
AWS
CodeCommit
Amazon
API Gateway
Amazon
Alexa
AWS IoT AWS Step
Functions

9. Lambda Execution Model
Synchronous
(push)
Asynchronous
(event)
Stream-based
Amazon
API Gateway
AWS Lambda
function
Amazon
DynamoDBAmazon
SNS
/order
AWS Lambda
function
Amazon
S3
reqs
Amazon
Kinesis
changes
AWS Lambda
service
function

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon API Gateway
Create a unified
API frontend for
multiple
microservices
Authenticate and
authorize
requests to a
backend
DDoS protection
and throttling
for your backend
Throttle, meter,
and monetize
API usage by
third-party
developers

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API Gateway Integrations
Internet
Mobile
apps
Websites
Services
AWS Lambda
functions
AWS
API Gateway Cache
Endpoints on
Amazon EC2
All publicly
accessible
endpoints
Amazon CloudWatch
Monitoring
Amazon
CloudFront
Any other
AWS service
Your
VPC
Endpoints
in your VPC
Regional API endpoints

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Step Functions
Easily coordinate multiple Lambda functions using visual
workflows
Visualize in the
console
Define in JSON Monitor
executions

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Benefits of Step Functions Orchestration
Change and add steps
without writing code
to evolve applications
and innovate faster
Coordinate and visualize
Lambda functions as a
series of steps to quickly
create serverless apps
Automatically trigger and
track each step at scale
and handle errors with
built-in retry and fallback
Productivity AgilityResilience

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s New in Serverless

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Serverless Application Repository
• Search and browse ready-made apps and samples
• Customize open-source apps to get started quickly
• Share apps privately or publically
• Monetize APIs using the AWS Marketplace
• All apps powered by AWS SAM

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s New in Serverless (cont.’d)
• Easy-to-use IDE in the cloud
• Built-in GitHub support
• Built-in Lambda blueprint support
• Built-in SAM Local:
• Local testing
• IDE debugging
• Deploy directly to Lambda or push
updates to GitHub
Cloud 9

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s New in Serverless (cont.’d)
• Lambda: Weighted aliases
• API Gateway: Substages for
incremental deployments
• AWS CodeDeploy: Automated
support for safe serverless
deployments
Deployments Enterprise connectivity
• Lambda: Concurrency metric
• Lambda: Per-function concurrency
throttles
• VPC integrations

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customers Love AWS Lambda

19. That’s a wrap
AI with AWS Lambda

21. Why photo book AI on AWS Lambda?
• Quiet periods followed by peaks in traffic
• Lot’s of photos to analyze for a single photo book
• Laziness: we do not want to manage servers

22. Challenge: size
• Models are trained on GPU machines on premise
• Complied on EC2 instances with (forked) Docker
Lambda image to reach optimum performance
• Trained models are pulled from S3 and cached for
reuse inside the Lambda container

23. Challenge: speed
• Average photo book contains 300 photos
• Multiple model executions per photo
• Thousands of inferences per photo book
• 35 minutes average execution duration in a non-
parallel environment
• We promise photo books within one minute

24. Solution: parallelism
• Controller
• Determines batches
• Invokes analysis Lambda per batch
• Bundles all results

25. Results
• We built our own Step Functions implementation
with parallelism
• Critical path is 10 seconds instead of 35 minutes
• Launched in March 2017, growth of 2000% in 2017
• No overhead
• Costs sublinear increase
• No staff upscale required
• Still hitting the 10 second mark

26. That’s a wrap
Without AWS Lambda, ReSnap would
not have been able to use deep
learning in production due to costs

27. Serverless DevOps / Deployment
Components

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Deployment Pattern Considerations
Minimizing Impact
to consumers
Rollback technique Execution model
factors
Deployment Speed

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Deployment Patterns
All at once
All traffic goes
from version OLD
to NEW at once.
Canaries/Linear
A small % of
production traffic is
sent to version NEW,
the remainder to
version OLD. After
some period of
waiting for
validation, traffic is
shifted incrementally
(with further
validation) or fully to
version NEW.
Blue/Green
Version NEW is
deployed and
tested against
before taking
production traffic.
After validated all
traffic goes from
version OLD to
NEW at once.

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Serverless Application Model (SAM)
CloudFormation extension optimized for
serverless
New serverless resource types: functions, APIs,
and tables
Supports anything CloudFormation supports
Open specification (Apache 2.0)
https://github.com/awslabs/serverless-application-model

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS SAM CLI (new!)
Work on your Serverless projects locally
Works with Lambda functions and “proxy-
style” APIs
Uses open source docker-lambda images
to mimic Lambda’s execution environment:
• Emulates timeout, memory limits,
runtimes
Debug Locally
https://github.com/awslabs/aws-sam-cli
Formally known as SAM Local
sam init --runtime python3.6

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lambda Alias Traffic Shifting
myLambdaFunction
1
2
3 = prod
4
5
6 = prod 5%
My First API
Stage variable = lambdaAlias
Prod
lambdaAlias = prod
aws lambda update-alias --name prod --function-name myLambdaFunction
--routing-config AdditionalVersionWeights={”6"=0.05}

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lambda Alias Traffic Shifting
myLambdaFunction
5
6 = prod
My First API
Stage variable = lambdaAlias
Prod
lambdaAlias = prod
aws lambda update-alias --name prod --function-name myLambdaFunction
--function-version 6 --routing-config ''

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM Globals + Safe Deployments
Globals:
Function:
Runtime: nodejs4.3
AutoPublishAlias: !Ref ENVIRONMENT
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
DeploymentPreference:
Type: Linear10PercentEvery10Minutes
Alarms:
# A list of alarms that you want to monitor
- !Ref AliasErrorMetricGreaterThanZeroAlarm
- !Ref LatestVersionErrorMetricGreaterThanZeroAlarm
Hooks:
# Validation Lambda functions that are run before & after traffic shifting
PreTraffic: !Ref PreTrafficLambdaFunction
PostTraffic: !Ref PostTrafficLambdaFunction
NEW!

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lambda Alias Traffic Shifting & AWS SAM
Alarms: # A list of alarms that you want to monitor
- !Ref AliasErrorMetricGreaterThanZeroAlarm
- !Ref LatestVersionErrorMetricGreaterThanZeroAlarm
Hooks: # Validation Lambda functions that are run
before & after traffic shifting
PreTraffic: !Ref PreTrafficLambdaFunction
PostTraffic: !Ref PostTrafficLambdaFunction
In SAM:
Note: You can specify a maximum of 10 alarms

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon API Gateway Canary Support
Use canary release deployments to gradually roll out new APIs
in Amazon API Gateway:
• configure percent of traffic to go to a new stage
deployment
• can test stage settings and variables
• API gateway will create additional Amazon CloudWatch
Logs group and CloudWatch metrics for the requests
handled by the canary deployment API
• To rollback: delete the deployment or set percent of traffic
to 0
NEW!

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Monitoring, Logging, Troubleshooting
Components

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudWatch Metrics
• 7 Built in metrics for Lambda
• Can call “put-metric-data”
from your function code
for custom metrics
• 7 Built in metrics for API-
Gateway
Metrics and logging are a universal right!
CloudWatch Logs
• Lambda Logging
• Custom logging from your
code with your language’s
equivalent of console.log()
• API Gateway Logging
• Custom formats
• Log Pivots
• Build metrics based on log
filters

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS X-Ray

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo
https://github.com/aws-samples/lambda-refarch-imagerecognition

43. Thank you!