Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Convert Your Code into a Microservice using AWS Lambda
Vyom Nagrani
Sr. Product Manager
July 13th, 2015
Agenda
• What is AWS Lambda?
• Lambda common use cases
• How Lambda works
• Building a microservice on Lambda
• Hands-on e...
What is AWS Lambda?
AWS Compute offerings
Lambda
Serverless compute
platform for stateless
code execution in
response to Triggers
ECS
Containe...
High performance at any
scale; Cost-effective and
efficient
No Infrastructure to manage
Pay only for what you use:
Lambda ...
How Lambda works
S3 bucket
notifications
DynamoDB
Streams
Kinesis
records
Cognito
sync
SNS
publish
Custom
triggers
CloudTr...
Old way of orchestrating workflows
SOURCE of
data
Fleet of servers poll for
changes
Listening to source Pushes to queue Pu...
New way of orchestrating workflows
SOURCE of
data
Attach a Lambda function
And that’s it!
Lambda does the
listening, the p...
Lambda common use cases
10
Typical usage scenarios
S3 + Lambda
Dynamic data ingestion
Image thumbnailing, Video
transcoding, File indexing, Log
pr...
Typical workflow for dynamic data ingestion using S3+Lambda
Notification
Amazon S3 AWS Lambda
processes
the object
Amazon ...
Customers running dynamic data ingestion and processing
using S3+Lambda
AWS
Lambda
Indexing
tables or
notifications
“I wan...
Typical workflow for real time stream processing using
Kinesis+Lambda
Amazon
Kinesis
AWS Lambda
processes
the stream
Amazo...
Customers running real-time data stream processing on
Kinesis+Lambda
AWS
Lambda
Aggregate
statistics
Real-time
analytics
K...
Typical workflow for smart monitoring and automation using
SNS+Lambda
AWS
Lambda
Amazon
SNS
Amazon
CloudWatch Alarm
Typical workflow for custom messaging using SNS+Lambda
Amazon
SNS
Amazon SNSAWS
Lambda
How Lambda works
Lambda functions
Simple resource model
• Set memory to any size from 128MB to 1GB, in 64MB steps
• Receive an equivalent p...
Writing and deploying Lambda Functions
• The Basics
– Node.js or Java
– AWS SDK comes built in and ready to use
– Lambda h...
Monitoring and debugging
Lambda Functions
• Console Dashboard
• Lists all Lambda functions
• Easy editing of resources, ev...
Invoking Lambda Functions
• Request Response: Call from mobile or web apps
– Wait for a response
– AWS SDK, AWS Mobile SDK...
Under the covers - Invocation permissions
• Resource policies
– Used in the Push model
– Define resource policies attached...
Building a microservice on Lambda
What are Microservices?
In short, the microservice architectural style is an approach to developing a single
application a...
What are Microservices? (Cont’d)
Microservices
Componentization via
Smaller Services
Decentralized
Governance
Organized ar...
Microservices and APIs
https://www.nginx.com/blog/building-microservices-using-an-api-gateway/
Sample Microservice on Lambda
https://aws.amazon.com/blogs/compute/the-squirrelbin-architecture-a-serverless-microservice-...
Hands-on exercises
Today’s exercises
1. Create and test your first Lambda function (Hello World)
2. Triggering Lambda from S3
3. Create a fil...
Q&A
Visit http://aws.amazon.com/lambda, the AWS Compute blog, and
the Lambda forum to learn more and get started using Lam...
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS
Convert Your Code into a Microservice using AWS Lambda
Upcoming SlideShare
Loading in …5
×

Convert Your Code into a Microservice using AWS Lambda

16,357 views

Published on

Convert Your Code into a Microservice using AWS Lambda.

Published in: Technology
  • Be the first to comment

Convert Your Code into a Microservice using AWS Lambda

  1. 1. Convert Your Code into a Microservice using AWS Lambda Vyom Nagrani Sr. Product Manager July 13th, 2015
  2. 2. Agenda • What is AWS Lambda? • Lambda common use cases • How Lambda works • Building a microservice on Lambda • Hands-on exercises • Q&A
  3. 3. What is AWS Lambda?
  4. 4. AWS Compute offerings Lambda Serverless compute platform for stateless code execution in response to Triggers ECS Container management service for running Docker on a managed cluster of EC2 EC2 Virtual servers in the Cloud
  5. 5. High performance at any scale; Cost-effective and efficient No Infrastructure to manage Pay only for what you use: Lambda automatically matches capacity to your request rate. Purchase compute in 100ms increments. Bring Your Own Code “Productivity focused compute platform to build powerful, dynamic, modular applications in the cloud” Run code in a choice of standard languages. Use threads, processes, files and shell scripts normally. Focus on business logic, not infrastructure. You upload code; AWS Lambda handles everything else. Why Lambda?
  6. 6. How Lambda works S3 bucket notifications DynamoDB Streams Kinesis records Cognito sync SNS publish Custom triggers CloudTrail activity LambdaDynamoDB Kinesis S3 Any custom Invoked in response to triggers - Changes in data - Changes in state Author in familiar language using any libraries; Execute only when needed, automatic scale Redshift SNS Access any service, including your own Any AWS Such as… “Lambda functions”
  7. 7. Old way of orchestrating workflows SOURCE of data Fleet of servers poll for changes Listening to source Pushes to queue Pull off queue Fleet of servers act as workers to process the data Auto-scale worker nodes to adjust with load S3 objects SNS messages Kinesis records DDB Tables IoT Devices … Cross AZ replication, Load Balancer
  8. 8. New way of orchestrating workflows SOURCE of data Attach a Lambda function And that’s it! Lambda does the listening, the polling, the queuing, the autoscaling, and spins up as many workers as needed to match the rate of change of source data
  9. 9. Lambda common use cases
  10. 10. 10 Typical usage scenarios S3 + Lambda Dynamic data ingestion Image thumbnailing, Video transcoding, File indexing, Log processing, Content validation, Aggregation and filtering Kinesis + Lambda Live stream processing Application activity auditing, transaction order processing, Click stream analysis, IoT device response, telemetry and metering SNS + Lambda Smart IT, Custom message Automating IT alarm response, Custom actions, IT auditing, customizing broadcast messages, text to email push Direct call + Lambda Serverless backend Microservices, Mobile backends, IoT backends DynamoDB + Lambda Database triggers [In preview] Data validation, Data filtering, Live notifications Alexa + Lambda Voice triggers Build a custom Automated Voice Response system in the cloud
  11. 11. Typical workflow for dynamic data ingestion using S3+Lambda Notification Amazon S3 AWS Lambda processes the object Amazon S3 New object uploaded Amazon DynamoDB
  12. 12. Customers running dynamic data ingestion and processing using S3+Lambda AWS Lambda Indexing tables or notifications “I want to apply custom logic to process content being uploaded to my data store”. • Watermarking / thumbnailing • Transcoding • Indexing and deduplication • Aggregation and filtering • Pre processing • Content validation Amazon S3 Bucket Events Processed files
  13. 13. Typical workflow for real time stream processing using Kinesis+Lambda Amazon Kinesis AWS Lambda processes the stream Amazon CloudWatch Logs Amazon SNS
  14. 14. Customers running real-time data stream processing on Kinesis+Lambda AWS Lambda Aggregate statistics Real-time analytics Kinesis Stream “I want to apply custom logic to process data being uploaded through my Kinesis stream”. • Client activity tracking • Metrics generation • Data cleansing • Log filtering • Indexing and searching • Log routing • Live alarms and notifications
  15. 15. Typical workflow for smart monitoring and automation using SNS+Lambda AWS Lambda Amazon SNS Amazon CloudWatch Alarm
  16. 16. Typical workflow for custom messaging using SNS+Lambda Amazon SNS Amazon SNSAWS Lambda
  17. 17. How Lambda works
  18. 18. Lambda functions Simple resource model • Set memory to any size from 128MB to 1GB, in 64MB steps • Receive an equivalent portion of other resources (disk, network, compute power, etc.) • Lambda tells you how much memory you used, so you can tune this setting. Flexible invocation paths • Lambda functions can be invoked “on demand” through CLI and Console • Subscribe to one or many event sources • Reuse the same Lambda function with multiple event sources Granular permissions control (using IAM) • Define what permissions the function has • Uses IAM role (execution role) for granular permission control • Recommended minimum permission – log to CloudWatch • E.g. “read from <X> DDB table only in the context of <Y> function”
  19. 19. Writing and deploying Lambda Functions • The Basics – Node.js or Java – AWS SDK comes built in and ready to use – Lambda handles inbound traffic – Use processes, threads, /tmp, sockets, … – Bring your own libraries, even native ones • Deployment options – Author directly using the console – Package code as a ZIP and upload through uploadFunction API – 3rd Party plugins (Grunt, Jenkins, CloudBees, Codeship) • Think stateless – Use S3, DynamoDB, or other Internet storage for persistent data – Don’t expect affinity to the infrastructure (you can’t “log in to the box”)
  20. 20. Monitoring and debugging Lambda Functions • Console Dashboard • Lists all Lambda functions • Easy editing of resources, event sources and other settings • At-a-glance metrics • Metrics in CloudWatch • Requests • Errors • Latency • Throttles • Logging in CloudWatch Logs
  21. 21. Invoking Lambda Functions • Request Response: Call from mobile or web apps – Wait for a response – AWS SDK, AWS Mobile SDK, REST API, CLI • Push Event: Incoming events from Amazon S3 or SNS – One event per Lambda invocation – Unordered model – 3 tries (won’t retry buggy code indefinitely) • Pull Event: Get DynamoDB changes or Amazon Kinesis records as events – Ordered model with multiple records per event – Unlimited retries (until data expires)
  22. 22. Under the covers - Invocation permissions • Resource policies – Used in the Push model – Define resource policies attached to a Lambda function – E.g. “User X can invoke on function Y in the context of bucket Z” – Resource policies allow for cross account access! • IAM roles – Used In the pull model – Lambda derives permission from execution role to read from particular Stream – E.g. “User A has permissions to read from Stream B in the context of Function C”
  23. 23. Building a microservice on Lambda
  24. 24. What are Microservices? In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler http://martinfowler.com/microservices
  25. 25. What are Microservices? (Cont’d) Microservices Componentization via Smaller Services Decentralized Governance Organized around Business Capabilities Smart endpoints and dumb pipes Infrastructure Automation Decentralized Data Management
  26. 26. Microservices and APIs https://www.nginx.com/blog/building-microservices-using-an-api-gateway/
  27. 27. Sample Microservice on Lambda https://aws.amazon.com/blogs/compute/the-squirrelbin-architecture-a-serverless-microservice-using-aws-lambda/
  28. 28. Hands-on exercises
  29. 29. Today’s exercises 1. Create and test your first Lambda function (Hello World) 2. Triggering Lambda from S3 3. Create a file deduplication Microservice – Use code from https://github.com/vyomnagrani/awslambda/blob/master/demo-s3-file-dedup.js 4. Triggering Lambda from SNS 5. Create a message customization Microservice – Use code from https://github.com/vyomnagrani/awslambda/blob/master/demo-sns-customize.js 6. Create a CRUD-backend Microservice with a public HTTP endpoint [Advanced users can jump directly to this one] – Build off of https://github.com/vyomnagrani/awslambda/blob/master/demo-CRUD-backend.txt
  30. 30. Q&A Visit http://aws.amazon.com/lambda, the AWS Compute blog, and the Lambda forum to learn more and get started using Lambda.
  31. 31. aws.amazon.com/activate Everything and Anything Startups Need to Get Started on AWS

×