Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons learned after a year of lambda - AWS Community Day SF 2017

879 views

Published on

Reach me on twitter - @matt_billock

Covers some of the lessons learned by Backand as we worked with AWS Lambda over a year in our multi-tenant serverless development platform

Published in: Technology
  • Be the first to comment

Lessons learned after a year of lambda - AWS Community Day SF 2017

  1. 1. Lessons Learned After a Year of AWS Lambda
  2. 2. ● AWS Lambda Overview ● Performance characteristics ● Lambda Integration ● Debugging AWS Lambda Agenda:
  3. 3. AWS Lambda Overview
  4. 4. AWS Lambda ● Created in 2014, launched at re:Invent ● Kicked off "serverless" (caveat) ● Function-as-a-Service ○Deploy code, not applications
  5. 5. AWS Lambda Details ● Supports Node.JS, Python, C#, and Java ● Can call to server functionality ● Code deployment options ○ Uploading as a ZIP file (S3 or HTTP) ○ In-browser editor
  6. 6. Calling AWS Lambda Functions ● Event-based triggers ○ On-Demand via HTTP ● Event Sources ○ AWS Services (S3, DynamoDB, others) ○ API Gateway ○ AWS SDK
  7. 7. Creating a Lambda Function (in pictures)
  8. 8. Demo
  9. 9. Demo
  10. 10. Demo
  11. 11. Demo
  12. 12. Demo
  13. 13. Demo
  14. 14. Performance Characteristics
  15. 15. ● Scraped logs of 2.6 million function calls ● Pulled from CloudWatch on AWS ○ Function execution only ● Data on computing resources consumed only Examining Lambda Performance
  16. 16. ● Graphs built on a sample of full data set (26,000 vs 2.6 million) ○ Sample is not random ● No relation to size of Lambda function ● Confounding factors may be present ● Significant outliers First, Caveats
  17. 17. Memory vs Execution Time
  18. 18. Memory vs Execution Time ● Does not incorporate HTTP Request ● Not a very strong correlation (r=-0.03) ● Three distinct groupings ● Tip: More memory allocated = faster processor
  19. 19. Runtime Histogram
  20. 20. Runtime Histogram ● Many functions run in < 100 ms ● Ignores code characteristics ● Ignores significant outliers on both ends ● Bimodal due to function characteristics
  21. 21. Improving the Analysis ● Capture full cycle times if possible ● Correlate runtimes with size of function ● Algorithmic complexity analysis ● Randomize sample selection
  22. 22. Lambda Integration
  23. 23. Four problems to solve: ● Problem 1 - Lambda Ownership ● Problem 2 - Deduplication ● Problem 3 - Triggering Lambda Functions ● Problem 4 - User Security Lambda Integration
  24. 24. Problem 1 - Lambda Ownership Option 1: User ownership ● Users host Lambda functions in their AWS account ● Backand uses provided credentials to call Lambda ● Maximizes freedom at the expense of ease of use (also, security)
  25. 25. Problem 1 - Lambda Ownership Option 2: Backand ownership ● Backand hosts Lambda functions in internal AWS account ● No need for user's AWS credentials ● Maximizes ease of use at expense of user freedom Solution selected: Backand owns code
  26. 26. ● Lambda function names need to be unique ● Name length limits: ○ 140 characters for full ARN ○ 64 character subset for function name Solution Selected: prepend guaranteed unique data ● Backand app names have a uniqueness constraint ● Could also use GUID, but this reduces name length to 32 characters Problem 2 - Deduplication
  27. 27. ● Most triggers AWS-based ○ S3, DynamoDB, Cloudwatch ○ None work for our use case ● API Gateway ○ Positive: Allows HTTP trigger ○ Negative: Complex configuration ○ Negative: Return value requirements Solution selected: AWS SDK Problem 3 - Triggering Lambda
  28. 28. ● Default: configure IAM profile and users ○ Need to give every dev access to AWS ○ Does not integrate with org security ○ Needs to be managed in multiple places ● Clients want flexibility and security ○ Need Single Sign-On ○ Need custom third-party security tools ○ Need ease of management Solution selected: Lambda Launcher Problem 4 - User Security
  29. 29. Debugging AWS Lambda
  30. 30. Debugging Lambda Functions ● Cloudwatch Integration ○ Provides call record, and console.log() ● Custom parameter input ○ Modal dialog presented before each run ● … and that's it
  31. 31. Emulating Lambda Locally Three things to mimic: ● Trigger Event ● Machine Environment ● Invocation
  32. 32. Emulating trigger event ● Each trigger sends different data ● Dump data from a sample call (console.log()) ● Keep in mind any transformations (API Gateway)
  33. 33. Lambda Machine Environment ● Based on Amazon Machine Instance ● Dynamically provisioned (hot vs cold) ● Temporary storage (/tmp, 500 MB) ○ NOT guaranteed to persist ● Cannot accurately model machine
  34. 34. Invoking Lambda ● External Function Call (entry point) ● Custom parameters in "event" object ● Execution context details in "context" ● Callbacks (if supported) in "callback"
  35. 35. Debugging Lambda Locally ● Execution environment hard to emulate ● Need to mimic input format from action ● Need to mimic output format ● Need a test harness
  36. 36. Creating a Test Harness (Node.js) ● Function Prototype ● Including Handler ● Mimicking Lambda Invocation ○ Parameters in event object ○ Handlers in the context object
  37. 37. Creating a Test Harness ● Calling a Lambda function
  38. 38. Test Harness Notes ● Callback mechanism ○ Need to adapt for each language ● Using callbacks in context parameter ○ Not supported, overloads context object ● Not a perfect substitute ○ Calls the function, but not from the same environment
  39. 39. What do we get? ● Per-line output via console.log ● Immediate feedback from command ● Breakpoints and other local debugging tools ● Unit and Integration tests ● CI/CD compatible (with CLI)
  40. 40. Deploying Your Code ● Zip-file from S3, or manual upload ○ Gotcha: zip only the source code, not the parent folder! ● Test upload ● Publish new API ● Complex, non-intuitive
  41. 41. Using Backand's CLI Tool ● Backand On-Demand Node.JS Lambda Functions ● Backand CLI abstracts away deployment ● npm install -g backand ● Requires registering with Backand
  42. 42. Lambda through a CLI ● Initialize a function ○ backand function init ● Copy your Node.JS code into created directory and iterate ● Deploy ○ backand function deploy
  43. 43. Calling Your Lambda ● Authentication ○ can be anonymous ○ api.backand.com/1/token ● cURL curl -H "<auth header>" "https://api.backand.com/1/function/general/<name>" ● JavaScript (with Backand SDK) backand.function.get(...)
  44. 44. What is Backand? ● Serverless app platform ● Manages your app's database ● Manages your app's security and authentication ● Provides custom server-side code execution ● Provides hosting options ● Provides logging, analytics, and more
  45. 45. Backand Features ● Automated REST API ● Bring your own Database ● Custom JavaScript Actions ● Server-Side Code Execution ● Batch and Bulk Processing ● Scheduled Tasks ● Automated messages ● Real-time Communications ● User and Role-based Security ● Social Media Authentication ● GUI Schema Editor ● Custom Queries ● REST API Playground ● Sample Code Generation ● Hosting ● Detailed Analytics ● Logging ● Multi-Platform SDK ● Single Sign On Support
  46. 46. ● Live online demo every other Wednesday ● Webinar - Invoking Lambda through Alexa Thursday, June 22nd, 11 AM Pacific Learn more at https://www.backand.com Want to know more?
  47. 47. THANK YOU https://github.com/backand matt@backand.com @matt_billock `

×