AWS Lambda: Best Practices and Common Mistakes - Chicago Cloud Conference 2020

AWS Lambda: Best Practices
and Common Mistakes
Given by Derek C. Ashmore
Chicago Cloud Conference
September 21, 2020
©2020 Derek C. Ashmore, All Rights Reserved 1

Who am I?
• Professional Geek
since 1987
• Java/J2EE/Java EE
since 1999
• AWS since 2010
• Azure since 2017
• Specialties
• Cloud
Workshops
• Cloud-native
Applications
• Yes – I still code!

Discussion Resources
• This slide deck
– https://www.slideshare.net/derekashmore/presentations
• Sample code on my Github
– https://github.com/Derek-Ashmore/
• Slide deck has hyper-links!
– Don’t bother writing down URLs

Agenda
The “What”
and “Why”
of AWS
Lambda
Code-Level
Tips
Operation
and Design
Habits
Summary /
Q&A
© 2020 Derek C. Ashmore, All Rights Reserved 4

What are AWS Lambdas?
• You provide custom code -> AWS runs it
– Java, Node.js, Python, Go, Ruby, .Net Core
– Can implement custom runtimes for languages not in the list!
• Computing power with less management
– AWS manages the hardware
– AWS autoscales that hardware
– AWS maintains that hardware
• Lambdas are event driven
– API Gateway (e.g. RESTful Web Service call)
– Many more
• Lambdas are stateless
• Not to be confused with “Lambda Expressions” in Java 8

Lambda Event Sources
• API Gateway
• SNS Messaging
Subscriptions
• Schedule
• Storage writes
– S3, DynamoDB, Kenesis
• SES Email receipt
• Cloudwatch
– Schedule, Events, log entries
• Cognito (Security)
• CloudFormation
– Creation script

What’s the Business Benefit
• Less Maintenance Hassle
• Unlimited Parallelism
• Current cost advantage
– Don’t pay for idle
– CPU cost currently lower
• Free tier
– 1 M executions and 400K compute seconds per month
– Memory allocated determines allowed free-tier runtime
• 20 cents per 1 M executions + memory/runtime cost
– Administration cost
• No O/S upgrades, server backups, etc.

There’s no free lunch
• Less control over environment
– Harder to tune
– Memory and time limits on execution
• Few Environment amenities
– No connection pooling, session support, caching
• Proprietary Interface
– Potential Technical Lock-in
• No Guarantee that AWS cost will be constant
– Potential Business Risk
• Modern version of CGI

Agenda
The “What”
and “Why”
of AWS
Lambda
Code-Level
Tips
Operation
and Design
Habits
Summary /
Q&A

What Makes a “Best Practice”?
• Makes Support Easier
• Increases Reuse
• Increases Performance
• Minimizes Resource Consumption
– Labor
– Runtime

Let’s start with Low-Hanging Fruit

Report Inputs/Env on Exception
• Place a Try / Catch in your handler
– Python Example
– Java Example
• Also check your arguments with a clear error message
def crossAccountHandler(event, context):
try:
………………
except Exception as e:
e.args += (event,vars(context))
raise

Check Arguments Up Front
• Check your arguments with a clear error message
– Python Example
– Java Example
def crossAccountHandler(event, context):
try:
if 'Assumed_Role' in event:
…………………
else:
raise Exception('Assumed_Role not provided as argument')
except Exception as e:

Specify Lambda Source Repo
• Explicitly put the source repository name in the Lambda comments
– In most organizations, the repository name isn’t obvious
– Others changing your code need it
– You don’t want source control to be out of date
"""
secretLambda.py
……………
Source Control: https://github.com/Derek-Ashmore/AWSDevOpsUtilities
"""

Separate Lambda from Business Logic
• Make business logic reusable
– Callable by other applications
– Usable on premises
• Easier to locally develop and debug
– Lambda-specific logic is thin!
def startStopHandler(event, context):
try:
executeStopStart(datetime.datetime.now()
, os.getenv('Scheduled_StartTime', ‘’)
, os.getenv('Scheduled_StopTime', ‘’)
, os.getenv('Scheduled_StartStop_Days', 'M,T,W,R,F’))
……………
return 0;

This is low-hanging fruit that will be appreciated by
your fellow developers!
• Log All Inputs and Environment on
Exception
• Check all arguments up front
• Document the source repo at the top.
• Repo readme can have other
developer specifics
• Separate Lambda code from business
logic
• Now let’s talk design and operations….

Agenda
The “What”
and “Why”
of AWS
Lambda
Code-Level
Tips
Operation
and Design
Habits
Summary /
Q&A

Automate builds and deployments!

Lambda Copies Everywhere!
• Changes / Bug Fixes need to be deployed everywhere
• Solving with automation solves the wrong problem!

One Copy for All!
• Scalable – only need to add accounts over time
• Bugfixes in one place
• Configuration usually in common DynamoDB table(s)
• Sample in Python here

Cross-Account Execution
• Algorithm is
– Assume a remote-account role using STS
• The response has temporary credentials
– Create a session using the remote account creds
– Do work in the remote account
• Example here: Derek-Ashmore/AWSDevOpsUtilities (Github)

For workloads over 15 min
• Executor that invokes lambda asynchronously for
each account
• Sample in Python here

Limit Custom Nesting to One Level
• Debugging with nested executions is
– Time consuming and difficult
– Can’t do locally
– Absolutely requires unique correlation id for the entire transaction
• Allows you to tell invocation history for one logical transaction
– Instead of deep custom nesting, use AWS Step Functions
• Use Step Functions if you need more

Nested Calls using AWS Step Functions
• AWS Step Functions
– Uses a State Machine model
• Think turn-style to get access to train
– States are “Locked” and “Unlocked”
– Locked → Payment input allowed, then “Unlocked”
– Unlocked → One person allowed through, then “Locked”
– Automatically provides correlation between invocations
• Unified logs for the entire transaction
– Now supported by X-ray (09/2020)
• Execution time and health per Step Function workflow

Operations and Design Habits
• Automate Builds and Deployments
• Only install Lambda’s Once
• Limit Lambda nesting to One Level
• Step functions if you need more
• Now let’s talk dependencies and secrets

Use Configuration Injection
• No environment specifics hardcoded in the Lambda deployment
• Use Environment Variables on the Lambda Definition
– No un-encrypted secrets (e.g. database password)
• Use Arguments in the triggering event
– No un-encrypted secrets
• Anti-Example
– Splunk forwarding Lambda with hard-coded Splunk channels

Providing Secrets to Lambdas
• Secrets are needed items like credentials of any type.
• Use IAM Roles to grant permission to read secrets
• Options are:
– Use KMS
• Encrypt credential and base64 encode it
– Place encrypted version in environment variable
• Sample Lambda and Encryption Script (here)
– Use a Digital Vault (e.g. AWS Secrets Manager)
• Sample Lambda here

AWS Secrets Manager
• Use IAM Roles to grant
permission to read secrets
• You don’t need a “secret” to
get a “secret”!

Avoid Heavy-Footprint Dependencies
• Minimizes load time
– Mitigates cold-start problem
• Java
– Use Guice over Spring
• Python
– Use AWS provided deps first (list is
here)

Idempotence
• Possible for your Lambda to be invoked multiple times for the same event
– Prevent repeat actions from having a different effect.
• Options
– Record the event id –> Skip repeated events
• Most event sources provide a unique request id
– Lambda invoking lambda does not!
• Negatively affects performance
– 1 extra read
– 1 extra write
• Need to roll-off old events
– Insure that the effect is the same each time
• Not perfect → You don’t control invocation order

Dependency Management
• AWS Layers can be used for
dependency management
– Can upgrade your library
dependencies in one location
• Provides convenience
• Use for custom runtimes!
• Beware of unintended
consequences
– Easies to inadvertently break
published lambdas using layers

Common Mistakes
• Deploying inappropriate workloads as Lambdas
– Install full Java web application as Lambda
• Required keep-alive lambdas to prevent cold-starts
• Not keeping business logic unit-testable
• Creating the distributed monolith
– Lambdas with extensive nested execution
– Lambdas not independently deployable

Further Reading
• This slide deck
– https://www.slideshare.net/derekashmore/presentations
• AWS Lambda Reading List
– http://www.derekashmore.com/2016/04/aws-lambda-reading-list.html
• Amazon’s Published Best Practice List
– https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html

Questions?
• Derek Ashmore:
– Blog: www.derekashmore.com
– LinkedIn: www.linkedin.com/in/derekashmore
• Connect Invites from attendees welcome
– Twitter: https://twitter.com/Derek_Ashmore
– GitHub: https://github.com/Derek-Ashmore
– Book: http://dvtpress.com/

AWS Lambda: Best Practices and Common Mistakes - Chicago Cloud Conference 2020

More Related Content

What's hot

Similar to AWS Lambda: Best Practices and Common Mistakes - Chicago Cloud Conference 2020

Recently uploaded

AWS Lambda: Best Practices and Common Mistakes - Chicago Cloud Conference 2020