SRV315 Building Enterprise-Grade Serverless Apps

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dr. Tim Wagner
General Manager, AWS Lambda and Amazon API Gateway
Cecilia Deng
Software Development Manager, AWS Lambda
SRV315
Building Enterprise-Grade
Serverless Apps
Satish Malireddi
Principal Architect, T-Mobile

Public Service Announcement:
Node 8.10 available in AWS Lambda
(And ICYMI: Go and .NET CoreCLR 2.0 are both GA!)

Agenda: 6 ways to improve your serverless mojo
1. Think in patterns: the serverless architecture playbook
2. Scale like the pros: understanding concurrency & throttling
3. SecOps like you mean it: permissions & privileges
4. Think in apps
5. Software lifecycle: from coding to deployment
6. When things go wrong: Error-handling and diagnostics
Guest Speaker: Jazz – Serverless Tools from T-Mobile

Key Trends
Don’t code it yourself!
The Rise of Managed Services
Stream processing and
“embarrassingly parallel” computing
Snowballing adoption of
microservices
Serverless, event-driven
architectures

1. Serverless Patterns:
Managed services + Lambda functions

Managed services as building blocks
Amazon SNS
Amazon SQS
Amazon S3
Messaging
Monitoring and Debugging
Storage
AWS X-Ray
AWS Lambda
Amazon API Gateway
Orchestration
API Proxy
Compute
AWS Step Functions
Amazon DynamoDB
Amazon Kinesis
Analytics
Database
Edge Compute
AWS Greengrass
Lambda@Edge
Amazon Athena
Amazon Aurora
Serverless (coming soon)

Patterns for the Cloud Era
• Media transform on upload: Amazon S3 event + Lambda
• NoSQL data cleansing: Amazon DynamoDB change streams + Lambda
• Serverless website: Amazon S3 + Amazon DynamoDB + Amazon API
Gateway + Lambda
• Click-stream analytics: Amazon Kinesis Data Firehose + Lambda
• Ordered event processing: Kinesis + Lambda
• Multi-function fanout: Amazon SNS (or Lambda) + Lambda
• Workflows: AWS Step Functions + Lambda
• Event distribution: Amazon CloudWatch Events + Lambda
• Serverless cron jobs: CloudWatch timer events + Lambda
• Login hooks: Amazon Cognito User Pools + Lambda
• GraphQL actions: AWS AppSync + Lambda

Patterns for the Cloud Era
• On-the-fly image resizing: AWS Lambda@Edge + Amazon CloudFront
• Email rules: Amazon SES + Lambda
• Configuration policy enforcement: AWS Config + Lambda
• Stored procedures: Amazon Aurora + Lambda
• Custom authorizers for APIs: API Gateway auth + Lambda
• DevOps choreography: CloudWatch alarms + Lambda
• Alexa skills: Amazon Alexa + Lambda
• Chatbots: Slack + Amazon Lex + Lambda
• IoT automation: AWS IoT + Lambda
• Smart devices: AWS Greengrass + Lambda
• On-prem file encrypt for transit: AWS Snowball Edge + Lambda

Meta-patterns
1. Service pushes async event to Lambda (S3, SNS)
2. Lambda grabs event from service (DynamoDB, Kinesis)
3. Synchronous exchange (Alexa, Lex)
4. Batch transform (Kinesis Firehose)
5. Microservice (API + Lambda + your choice of DB)
6. Customization via functions (AWS Config, SES rules)
7. Data-driven fanout (S3-Lambda, Lambda-lambda)
8. Choreography (Step Functions + Lambda)
9. Lambda functions in devices (Greengrass, Snowball Edge)

Coming Soon:
SQS as a built-in Lambda event
source

Patterns: Best practices
• Don’t reinvent the wheel – use managed services when possible
• …and stay current 
• Lambda@Edge expands support for headers with S3 origin events
• Keep events inside AWS services for as long as possible:
• Example: S3  Lambda  client is preferable to S3  client  Lambda
• Verify limits end-to-end
• Prefer idempotent, stateless functions, and when you can’t…
• Use Step Functions if you need stateful control (retries, long-running)
• Building a SaaS or ISV platform? Use the techniques AWS uses:
1. Architect it serverlessly using Lambda and API Gateway
2. Offer your ecosystem customization via functions
3. Enable customers and partners to build reactive systems by emitting events

2. Concurrency, Throttling, &
Timeouts

API Gateway Throttling
Three levels of throttling for APIs:
API Key level throttling – configurable in usage plan
Method level throttling – configurable in stage settings
Account level throttling – limits can be increased

API Gateway Throttling
Token bucket algorithm
Burst – the maximum size of the bucket
Rate – the number of tokens added to the bucket
You can also set a daily, weekly, or monthly quota.

Lambda Concurrency Management
Account level safety limit: 1,000 (we will happily raise it )
Per-function concurrency limits
• You set these
• Optional: all functions without them share the remaining pool
• Allows you to protect the capacity of a production system (but
there’s no provisioning charge if you don’t use it)
• Also good for:
• Keeping Lambda’s scale from overwhelming a legacy system
• Preventing a runaway function while you’re developing/testing
• Limiting your costs

Lambda Burst & Scale Throttles
Instantaneous burst limits:
• In US-EAST-1, EU-WEST-1, and US-WEST-2: 3,000 concurrent
• All other regions: 1,000 concurrent
• Thereafter:
• 500/minute additional concurrent (until you reach per-function or
account limits)

Max Timeouts
API Gateway: 30 seconds
Lambda: 5 minutes
Lambda@Edge: 30 seconds (origin), 5 seconds (view events)
What do you do when you want to trigger a function that
needs more than 30 seconds? Go async:
1. API  Step Function  your function
2. API  f1 (sync)  f2 (async)

3. Security and AuthZ

Key Questions for Securing Serverless
Architectures
What can call this function / API?
What can this function / API call?
Who can change these decisions?
How can I know what’s happening?

Key Tenets for Securing Serverless Architectures
What can call this function / API?
As few things as possible
What can this function / API call?
As few things as possible
Who can change these decisions?
As few people as possible
How can I know what’s happening?
Audit!

3 Simple Concepts
1. Resource Policies
Analogy: Who would I invite into my
house?

3 Simple Concepts
2. Execution Roles
Analogy: What do I let my kids play
with?

3 Simple Concepts
3. Roles
Analogy: Am I at work or at home? ?

• Resource Policies for APIs who can call
the API, and under what conditions
• Sample Uses
• Enable users from a different AWS
account to securely access your API
• Enable customers to restrict access to
specific IP Address ranges or CIDR
blocks
• Limit access to a specific Lambda
function
New API Gateway Feature - Resource Policies

API Gateway: Resource policies vs. Custom
Authorizers
• Which should I use?
• If resource policies can express the restriction, use them.
(They’re free, fast, and built in.)
• How do these two features interact?
• We evaluate in three phases; all 3 have to pass:
• Pre-authN (time checks, source IP checks)
• Post-authN (call principal checks)
• Custom authorizer (if present)
• We stop as soon as permission is denied, so we only run your
custom authorizer if it’s necessary to make a decision.

API Gateway: Handy Auth Comparison Chart
Feature AWS_IAM Token Request
Amazon
Cognito
Authentication X X X X
Authorization X X X
Signature V4 X
Amazon Cognito User
Pools
X X X
Third-Party
Authentication
X X
Multiple Header
Support
X
Additional Costs None Pay per
authorizer
invoke
Pay per
authorizer invoke
None

Roles and X-account Calls
• Lambda execution roles control what the function can do.
• API Gateway invocation roles let you change the calling
principal to any role/user in the same account.
• Stick to API Gateway as the principal unless you have a specific
need for this
• API Gateway now also supports:
• Cross-account Lambda invocations
• Cross-account Lambda custom authorizers
• Useful pattern: Switch accounts inside an API or function:
Kinesis stream in Account A  function (Account A)  API (Account B)

Keeping track: Auditing and Config Policies
• Use AWS CloudTrail to track API and function management
(creation, deletion, retrieval, etc.)
• Can also audit function invocation for Lambda
• Run queries against CloudTrail data using Amazon Athena
• Or build a simple analysis engine using Lambda functions
• Use AWS Config to continuously monitor and enforce
policies
• For example: Enforcing resource-level scoping on Lambda
functions or custom authorizers for APIs

Calling Third Party Services Securely
AWS Secrets Manager
• Securely store, retrieve, and manage database credentials,
API keys, and other secrets for AWS or third-party systems
• Rotate, audit, and version secrets
• Built-in support for MySQL, PostgreSQL, and Amazon
Aurora on Amazon RDS
• Supports both default and custom KMS keys
• Avoid cleartext secrets in Lambda env vars or code!

4. It’s All about [Serverless] Apps

AWS Serverless Application Repository
• Discover, customize, and deploy serverless apps
• Publish and share apps…
• Within your AWS account
• Across accounts
• With everyone (public access)
• ICYMI: Now GA!
• Finance apps based on NumPy library
• Roadmap: Look for more app-centric features in the
Lambda console and fast iteration on the app repo

Growing Set of App Producers

Apps are based on SAM (Serverless App Model)
• ICYMI: SAM added more API Gateway support in March:
• Logging and metrics configuration
• CORS
• Regional endpoints
• Binary data
• Method-level settings like throttling limits and cache TTL
• Amazon CloudWatch Logging event source support

Open Source Announcement
SAM implementation is now open source!
Help us build SAM tools by contributing to the open SAM
specification or the SAM translator:
• Add event sources
• Embed SAM transformations in other tools

5. Building, Testing, and Deploying
Serverless Apps

AWS CodePipeline
AWS Code Suite
AWS
CodeCommit
or GitHub
AWS
CodeBuild
(build)
AWS
CodeBuild
(test)
CloudFormation
(deploy)
AWS
CodeDeploy
AWS Cloud9
Editor
AWS
CodeStar

Best Practices for Serverless Development
• CI/CD pipeline for apps (including repo publishing!)
• Test and debug locally
• ICYMI: Python debugging in Cloud9 IDE
• AWS CloudFormation can deploy to multiple regions
• Useful for multi-region APIs
• Deploy incrementally for safety
• Weighted aliases in Lambda
• Canary stages in API Gateway
• AWS CodeDeploy can automate SAM app deploys (including
rollback)

6. When Things Go Wrong:
Error-Handling and Diagnostics

API Gateway CloudWatch Metrics
API Gateway Default metrics (free):
Count total number of invokes received by API Gateway
4XXError #invokes that generated a 4XX error
(includes throttling)
5XXError number of invokes that generated a 5XX error
Latency total time to fully process request
IntegrationLatency time API Gateway took to call integration
CacheHitCount number of successful cache fetches
CacheMissCount number of unsuccessful cache fetches

API Gateway CloudWatch Metrics
Included for free
Granularity: per-API stage
Default Metrics
CloudWatch pricing
Granularity: per-method
Can be globally enabled
Detailed Metrics

Lambda CloudWatch Metrics (free)
At both function and version level:
• Invocations, Errors, Throttles, IteratorAge, DeadLetterError
At both account and function level (when limit is set):
• ConcurrentExecutions
At the account level only:
• UnreservedConcurrentExecutions

Amazon CloudWatch Alarms
Any metric can be tied to an alarm
Alarm notifications can be sent to Amazon SNS topic
SNS topic can then send to any number of destinations
Email address
SQS queue
Lambda function

API Gateway CloudWatch Logs
API Gateway Logging
Two levels of logging, error and info
Optionally log method request/body content
Set globally in stage, or override per method
API Gateway Access Logging
Customizable format for machine parsable logs
Log Pivots
Build metrics based on log filters
Jump to logs that generated metrics

Lambda CloudWatch Logs
Basics:
• Automatically configured
• Basic info (requests, duration, memory consumption,
etc.) handled automatically
• Application can add log entries easily
Cool Stuff:
• Grab a range on the console graph and jump to the
logs for that time range
• Send logs to another Lambda function
• Use Elasticsearch & Kibana to aggregate/analyze
• “Top talkers”

The Lambda Function Request Lifecycle
1. Download your code (as a SquashFS filesystem)
2. Start the language runtime
3. Load your code (e.g., class loading in Java)
4. Run “init” code (e.g., static initializers in Java)
5. Process the request warm

Same View in AWS X-Ray

Things You Can Track in X-Ray
• Cold vs warm starts
• Async dwell time (time event spends in queue)
• Duration of calls to other AWS services (and with a little
work to add trace points, third-party services)
• Errors and throttles from AWS service calls
Lots of great third-party offerings can also help!

Making Cold Starts More Efficient
• Control the dependencies in your function's deployment
package
• Tree shake your Java code! (Hint: Spring is expensive )
• Can combine “hot paths”…but preserve agility
• Optimize for your language, where applicable…
• Node – Browserfy, Minify
• AWS Java v2 SDK (preview)
• Only use VPC if you need access to a resource there!
• Delay loading where possible

Summary: 6 ways to improve your serverless
mojo
1. Think in patterns: the serverless architecture playbook
2. Scale like the pros: understanding concurrency & throttling
3. SecOps like you mean it: permissions & privileges
4. Think in apps
5. Software lifecycle: from coding to deployment
6. When things go wrong: Error-handling and diagnostics

Introducing Jazz
T-Mobile’s Serverless Development Platform
https://github.com/tmobile/jazz
Satish Malireddi
Principal Architect, T-Mobile

▪ As America's Un-carrier, T-Mobile
US, Inc. is redefining the way
consumers and businesses buy
wireless services through leading
product and service innovation.
▪ NASDAQ traded public company –
TMUS
▪ Operating two flagship brands: T-
Mobile and MetroPCS
▪ Based in Bellevue, Washington
About T-Mobile

Serverless computing allows
you to build and run
applications and services
without thinking about servers
• No server management
• Scales on demand
• High availability
• Don’t pay for idle capacity
Serverles
s…
Jazz…
 A Serverless Development
Platform
 Jazz provides developers a fast
on-ramp to build & manage their
serverless applications
 It is not another FaaS implementation.
Rather, it enhances the usability of
existing FaaS systems
 Jazz has well-built interfaces designed
to let developers quickly self-start and
focus on code

Serverless Use Cases at T-Mobile...
 APIs/Microservices
 Static websites/single page
applications
 Mobile backends & serverless data
stores
 Timer-based processing (cron jobs)
 Event-based processing
 S3 upload
 Payment failed
 Monitoring alert triggered
 Real-time stream processing
 IoT events

Our Serverless Journey…
 Cloud
Operations
 Proof of
concepts
 MobileLife
 Microsites
 Policy & compliance
Manager
 Jazz platform
Open source!
2016
2017
2018
 3000+ Lambda functions
 100+ API gateways
 25+ Applications that are
live
 Millions of invocations/day
 Alexa skills & slack bots

Initial set of challenges faced…
 Packaging & deploying (CI/CD)
 Multi-tenancy
 Local testing
 Logging & monitoring
 Service sprawl
 Secret management
 Security, auditing & best practices
 Integration with enterprise services

Now, how can we drive adoption within T-
Mobile?
 Serverless is just like cloud 6 years ago. We’ve seen that cloud
worked and we’ve realized that serverless works!
 FaaS is great, but can we build production ready applications &
operate them at scale?
 Can we make it simpler for enterprise users to use serverless?
 How can we drive adoption internally in T-Mobile?

Focus on code, Jazz takes care of everything
else!
Jazz – Our solution to drive serverless
adoption
CI/CD
Multi Environment
Compliance
Monitoring
Security
Multi-Tenancy
Code Templates
LoggingManagement &
Control
https://github.com/tmobile/jazz

Jazz Architecture
Jazz itself is
serverless—and
written in Jazz!

Jazz Components
Jenkins:
• CI/CD
• Package & deploy
• Code & Dependency scan
• Test integration
Bitbucket/Gitlab:
• NodeJS/Python/Jav
a/Go*
• Commit hooks for
CI/CD
• Code/Swaggers
CloudFront
S3 DynamoDB
AWS KMSAmazon ES
API Gateway AWS Lambda
Kinesis

How Jazz helped us with adoption within T-
Mobile Serverless microservices are being created in minutes
 Applications are going live without teams managing a single server
 Active users exploring serverless increasing each month
 3,000+ Lambda functions deployed & managed via the platform
 Code templates made it easy to share best practices
 CI/CD, compliance & security comes with platform
 No server patching & significant cost reductions

What’s Next…
 Serverless - First approach for cloud native applications
 Open source - First development for Jazz & community
engagement
 Support new serverless patterns, service types & runtimes
 Enhanced Security Controls
 API Versioning, Rollbacks, B/G & Canary deployments
 Serverless Application Repository Integration
 Local testing and debugging
 What do you need?

GitHub: https://github.com/tmobile/jazz
OSS: https://opensource.t-mobile.com
Slack: https://tmo-oss-getinvite.herokuapp.com/
Email: serverless@t-mobile.com
Try Jazz: http://try.tmo-jazz.net
• For Reg. Code, Slack/Email us
Helpful Stuff…

Try it yourself…
http://try.tmo-jazz.net
For Reg. Code, slack/email us!
T-Mobile Confidential

Speaker Info / Social Media Channels
t: timallenwagner@ (including DM)
aws.amazon.com/lambda
aws.amazon.com/api-gateway
serverlessrepo.aws.amazon.com/applications
aws.amazon.com/blogs/compute
AWS Forums

Please complete the session survey in
the summit mobile app.
…and…Go Serverless!!

Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session
you attended.
3. Tap Session
Evaluation to submit your
feedback.

Thank you!

SRV315 Building Enterprise-Grade Serverless Apps

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to SRV315 Building Enterprise-Grade Serverless Apps

Similar to SRV315 Building Enterprise-Grade Serverless Apps (20)

More from Amazon Web Services

More from Amazon Web Services (20)

SRV315 Building Enterprise-Grade Serverless Apps