Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Walk in the Cloud with AWS Lambda


Published on

AWS Lambda is a new compute service that runs your code in response to events and automatically manages compute resources for you. In this session you’ll learn what you need to quickly begin building applications that use AWS Lambda as a serverless back-end. We’ll cover key Lambda features, its programming model, key scenarios, and tips on getting the most out of Lambda functions.

Published in: Technology
  • Be the first to comment

A Walk in the Cloud with AWS Lambda

  1. 1. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved A Walk In the Cloud with AWS Lambda Tara E. Walker AWS Technical Evangelist @taraw
  2. 2. AWS Compute offerings Lambda Server-less compute platform for Stateless, event-driven code execution EC2 Virtual servers in the cloud
  3. 3. High performance at any scale; Cost-effective and efficient No Infrastructure to manage Pay only for what you use: Lambda automatically matches capacity to your request rate. Purchase compute in 100ms increments. Bring Your Own Code “Productivity focused compute platform to build powerful, dynamic, modular applications in the cloud” Run code in a choice of standard languages. Use threads, processes, files and shell scripts normally. Focus on business logic, not infrastructure. You upload code; AWS Lambda handles everything else. Why Lambda?
  4. 4. Why Lambda? • You want to thumbnail images as they arrive in an S3 bucket. • You’d like to check that every address stored in Amazon DynamoDB is well formed. “Let’s Examine a Simple Scenario”
  5. 5. Why Lambda?: • Provision a fleet of proxy machines to capture uploads. • For each upload, enqueue a job to process it. • Provision a second fleet of machines to read and process jobs. • Pick a deployment solution. • Plan capacity, accounting for fault tolerance, good long- term utilization, burst capability, etc. • Monitor 24x7x365 for capacity, health, security, etc. • Migrate to new instance types over time, keeping OS and language runtimes patched and up to date. “For a Simple Scenario there can be complicated solutions”
  6. 6. The Why of Lambda…. What if there were a better way? Easy to author Easy to scale Easy to manage Easy to maintain Easy to pay for Easy to deploy Easy to maintain
  7. 7. What is Lambda? • Event-Driven Compute in the Cloud • Launched last November at re:Invent – Lambda functions: Stateless, request-driven code execution – Triggered by events (state transitions) in other services: • PUT to an Amazon S3 bucket • Write to an Amazon DynamoDB table • Record in an Amazon Kinesis stream • Message in an Amazon SQS queue • Transition in an Amazon EC2 instance • Goal: Any API call or resource transition – Makes it easy to… • Transform data as it reaches the cloud • Perform data-driven auditing, analysis, and notification • Kick off workflows
  8. 8. AWS Lambda or EC2 / ECS? AWS Lambda • Request-driven – Events – Data Triggers • Prioritizes ease of use – One OS – Default hardware choice • AWS owns and manages the infrastructure for you • Implicit scaling; just make requests Amazon EC2 and ECS • Infrastructure rental – Pre-configured AMIs – Build Custom AMIs • Flexible: – Choose instance type – OS – Language, etc.. • You own and configure the infrastructure • Scale by provisioning instances or containers
  9. 9. Lambda Under the Covers
  10. 10. How Lambda works S3 event notifications DynamoDB Streams Kinesis events Cognito events SNS events Custom events CloudTrail events LambdaDynamoDB Kinesis S3 Any custom Invoked in response to events Author in familiar language using any libraries; Execute only when needed, automatic scale Redshift SNS Access any service, including your own Any AWS Such as… “Lambda functions”
  11. 11. Writing Lambda Functions • The Basics – Stock node.js – AWS SDK comes built in and ready to use – Lambda handles inbound traffic • Stateless – Use S3, DynamoDB, or other Internet storage for persistent data – Don’t expect affinity to the infrastructure (you can’t “log in to the box”) • Familiar – Use processes, threads, /tmp, sockets, … – Bring your own libraries, even native ones
  12. 12. Configuring Lambda functions Simple resource model • Set memory to any size from 128MB to 1GB, in 64MB steps • Receive an equivalent portion of other resources (disk, network, compute power, etc.) • Lambda tells you how much memory you used, so you can tune this setting. Flexible invocation paths • Lambda functions can be invoked “on demand” through CLI and Console • Subscribe to one or many event sources • Reuse the same Lambda function with multiple event sources Granular permissions control (using IAM) • Define what permissions the function has • Uses IAM role (execution role) for granular permission control • Recommended minimum permission – log to CloudWatch • E.g. “read from <X> DDB table only in the context of <Y> function”
  13. 13. Under the covers – Lambda Functions • Invoke/Call from mobile or web apps – Wait for a response or just send an event and keep going – AWS SDK, AWS Mobile SDK, REST API, CLI • Send events from Amazon S3 or SNS: – One event per Lambda invocation – Unordered model – 3 tries (won’t retry buggy code indefinitely) • Process DynamoDB changes or Amazon Kinesis records as events: – Ordered model with multiple records per event – Unlimited retries (until data expires)
  14. 14. Under the covers - Invocation models • Push model - Invoke API call – “RequestResponse” mode returns a response immediately – “Event” mode puts function call into a queue, picked up by a poller and then invoked; Used by S3 – Caller (e.g. S3) derives permission through resource policies on Lambda function • Pull model - Subscribe to event source – points poller at a particular data stream; Used by DDB/Kinesis – Poller derived permission from execution role to read from particular DDB update stream
  15. 15. Under the covers - Invocation permissions • Resource policies – Used in the Push model – Define resource policies attached to a Lambda function – E.g. “User X can invoke on function Y in the context of bucket Z” – Resource policies allow for cross account access! • IAM roles – Used In the pull model – Lambda derives permission from execution role to read from particular Stream – E.g. “User A has permissions to read from Stream B in the context of Function C”
  16. 16. Monitoring and debugging Lambda Functions • AWS Lambda console includes a dashboard for functions • Lists all Lambda functions • Easy editing of resources, event sources and other settings • At-a-glance metrics • Metrics automatically reported to Amazon CloudWatch for each Lambda function • Requests • Errors • Latency • Throttles • Logs captured by Amazon CloudWatch Logging service
  17. 17. Real time data processing using Kinesis • Kinesis - A managed service for streaming data ingestion and processing – Data shows up as ordered stream of events supporting multiple readers • All data is stored for 24 hours – Streams are made of Shards • Scale Kinesis streams by adding Shards – Each Shard ingests data up to 1MB/sec and emits data up to 2MB/sec • Using Lambda with Kinesis – Concurrency = Number of Kinesis shards • Increasing shards will cause more Lambda functions invoked concurrently – Batch size = payload size read per shard • Each function call reads serially from a shard • Control how many events are processed per invocation • Increasing batch size will cause fewer Lambda function invocations with more data processed per function
  18. 18. Demo: Hello Lambda
  19. 19. Lambda usage scenarios
  20. 20. Scenario #1 - Dynamic data ingestion: Amazon S3 “I want to apply custom logic to process content being uploaded to my data store”. • PDF watermarking • Image thumbnailing and transcoding • Document metadata Indexing • Log aggregation and filtering • RSS feed processing • Media content validation
  21. 21. Data Triggers: Amazon S3 Amazon S3 Bucket Events AWS Lambda Original image Thumbnailed image 1 2 3
  22. 22. Data Triggers: Amazon DynamoDB AWS LambdaAmazon DynamoDB Table and Stream Send Amazon SNS notifications Update another table
  23. 23. Audit and Notify AWS API calls AWS CloudTrail Logs AWS LambdaS3 Bucket events Amazon SNS notifications
  24. 24. Additional scenarios Cognito CloudFormationLambda Lambda Validate cross device sync data Customized Push notifications SNS Lambda Custom deployment rules Smart workflows (Github, CloudWatch alarms) LambdaDynamoDB DB Triggers (in preview)
  25. 25. Custom Events AWS LambdaInsert YOUR App Event Here
  26. 26. AWS Lambda now Generally Available
  27. 27. Production Release of AWS Lambda • Larger default limits – 100 concurrent executions – 1,000 invokes per second – Increases available via AWS customer service • Preview label removed – Updated API based on feedback during preview – Multiple Lambda functions per stream – Easier to use programming model
  28. 28. New Features
  29. 29. Mobile Compute: Building Backends Launching Mobile Compute • Request/response • AWS Mobile SDK – iOS – Android • Easy Personalization …for devices …for end users AWS LambdaMobile App
  30. 30. Mobile backends • Synchronous invocations – Lambda will return a response as soon as the function finishes executing (JSON in, JSON out) • Mobile SDK integration (Android, iOS) – Can use @LambdaFunciton to annotate app methods and map them to Lambda function calls • Additional information in “context” – When invoked through the mobile SDK, access to information about the device, app and Cognito identity. Client context includes appVersion, appBuild, appPackageName, appName,devicePlatformVersion, devicePlatform, deviceManufacturer,deviceModel,deviceLocale
  31. 31. New AWS Event Sources for Lambda • Amazon Simple Notification Service (SNS) – Easily target, route, transform, filter, or audit messages – Trigger an AWS Lambda function by sending it notifications – Turn Amazon CloudWatch alarms into actions Lambda FunctionSNS
  32. 32. New AWS Event Sources for Lambda • Amazon Cognito – Before: Easily synchronize user data across their devices – Now: Take action when that data changes – Examples: Verify and respond to game state updates Lambda FunctionAmazon Cognito
  33. 33. New Authorization and Auditing Features • Easier setup and configuration – No need for invocation roles – Use resource policies to enable senders to call your function • Cross-account access support – Securely grant execution access to third parties – Send events from S3 buckets in a different AWS account • AWS CloudTrail integration – Track AWS Lambda API calls in AWS CloudTrail logs – Audit access logs easily with a Lambda function
  34. 34. Improved Metrics and Diagnostics • Discover and take action when you reach your concurrent execution limits – New Amazon CloudWatch throttling metric • Find relevant log entries faster – Sort CloudWatch Logs by time of last entry – See creation time in log stream names and easily filter on it – View the target log stream in your Lambda function
  35. 35. Java • You can already call Java programs from Lambda functions today… – Java and other languages are automatically included in your filesystem view…don’t wait to start using them! – Freezing ensures you don’t pay repeatedly for JVM boot • We’ll make this even easier with built-in support for AWS Lambda functions written in Java.
  36. 36. How can you use these features? “I want to send customized messages to different users” SNS + Lambda “I want to send an offer when a user runs out of lives in my game” Amazon Cognito + Lambda + SNS “I want to transform the records in a click stream or an IoT data stream” Amazon Kinesis + Lambda
  37. 37. Demo: Building a Mobile Backend with AWS Lambda
  38. 38. Three Next Steps 1. Go to the AWS Management Console to create and test your first Lambda function. The first 1M requests each month are on us! 2. Use the AWS Mobile SDK and Lambda to quickly create an instantly scalable mobile app. 3. Use AWS Lambda to add custom logic to S3, DynamoDB, SNS, Amazon Kinesis, or Amazon Cognito events…no servers required!
  39. 39. Next steps • Lambda console based walkthroughs – Hello World – Log Amazon S3 update to CloudWatch – Log Amazon Kinesis stream update to CloudWatch – Build a mobile backend • Lambda CLI based walkthroughs – Hello World – Generate thumbnails from S3 uploads – Log Amazon DynamoDB update to CloudWatch logs – Log Amazon Kinesis stream update to CloudWatch – Detect patterns in CloudTrail logs using S3
  40. 40. Visit, the AWS Compute blog, and the Lambda forum to learn more and get started using Lambda. Q & A
  41. 41. Thank You @taraw