1. Who Am I ?
By day:
Technical Program Manager
• National Rural Electric Cooperative Association
• Large 40M+ Angular, Backbone.JS, .Net WebApi
• application
By night
JavaScript Extraordinaire
Co-Founder & Chief Engineer for Hoozip.com
• Angular
• AWS S3
• AWS Lambda
• AWS Api Gateway
• AWS EC2
@javascriptbully
2. • Mongo
• Express
• Angular
• NodeJS
• AWS
• NodeJS
• Angular
• Lambda
@javascriptbully
“Time to get Mean!”
?
Who Am I ?
4. Event-Driven Compute
• As data enters your cloud infrastructure Lambda makes it extremely easy to
transform that data by applying your function to the data based on events you
define.
• Examples include data transformation, image transformation, large scale data
analysis. For example, run transformation functions whenever a DynamoDB
table is loaded.
• Ability to scale horizontally across large scale work loads and dynamically then
instantly shrink back once the work is complete.
• 100 concurrent executions
• 1,000 invokes per second
Quickly build mobile Backends
• AWS mobile SDKs
5. Lambda offering overview
• Runs code in response to events
• Object uploads to Amazon S3
• Updates to Amazon DynamoDB via Streams
• Data in Amazon Kinesis Streams
• In app activity
• Lambda handles all capacity, scaling, patching and infrastructure administration
• Lambda only runs your code when triggered so you don’t have to pay for unused capacity
• Provides realtime metrics and logs to Amazon CloudWatch
• Low cost, No upfront costs
• Code is charged in measurements of 100ms increments
• Python, Java, NodeJs
• Lambda Function (Micro service architecture)
• Prebuilt examples
• Built in support for AWS SDK
• AWS Free Tier you can try Lambda for free
• Security is baked in
6. Lambda offering overview (cont’d)
AWS Lambda supports the following runtime versions
• Node.js: v0.10.36
• Java: Java 8
• Python: Python 2.7
If you author your Lambda function code in Node.js,
the following libraries are available in the AWS Lambda
execution environment so you don't need to include
them:
• ImageMagick: Installed with default settings. For
versioning information, see imagemagick nodejs
wrapper and ImageMagick native binary
(search for "ImageMagick").
• AWS SDK: AWS SDK for JavaScript version
2.2.32
If you author your Lambda function code in Python, the
following libraries are available in the AWS Lambda
execution environment so you don't need to include
them:
• AWS SDK for Python (Boto 3) version 1.2.3
There are no additional libraries available for Java.
7. Hoozip Use Case
• Want to send property reports on demand via SMS
• Wanted to follow an event driven, micro service design
• Scale our systems up or down when certain load metrics are hit
• Send out notifications to owners of properties when their property is requested
• Managing and reacting to all those events would require a complex infrastructure, so
often we simply put them together in one controller action or use observers that run in
the same processes as our applications. This makes the codebase more complex as
parts start getting interwoven.
• Most teams start pushing those tasks into background workers, but the infrastructure
necessary for managing tasks this way is overhead too. Therefore, background
workers are typically limited to the most important tasks. This is especially true when
they don’t get automatically triggered by events, but need to be triggered through the
codebase. Doing this adds another level of complexity to the code in order to
understand which part triggers which event.
• It lets you write small NodeJS functions that will be called with the event metadata
from events triggered by various services or through your own code.
8. Hoozip
• Hoozip.com is a marketplace and software provider for real estate investors
(rehabbers, wholesalers, lenders) to find, post, and curate real estate
investment deals in an efficient way.
9. What we normally would do
• Size Provision Scale Servers
• Estimate capacity
• Run fault tolerance
• Manage Operating System
Updates
• Apply Security Patches
• Monitor for performance and
availability
• Lambda does the
impedance matching for
your event flow so you don’t
have to worry about over or
under provisioning as
Lambda scales elastically
10. call twilio
post to ApiGateway
call Lambda
parse message
fuzzy address
geoCode address
call property API
call bitly API
persist to DynamoDB
generate message
post back to twiliosend message
How we did it…
* As the API get’s busier, we’ll add a queueing / push
notification mechanism via SQS & SNS
12. Hoozip Architecture
• We serve our Angular app out of a S3 Bucket
• We use CloudFront as our CDN for all static assets including
the Angular App
• Most of our services are separate stand alone projects and
handled via Api Gateway and Lambda
• We currently use DynamoDB for persisting and Postgres as
our store for financial related actions and some GeoSpatial
calculations.
• When we can’t use Lambda (static IP addressing,
computationally heavy work loads) we use EC2 with Node
v4.0, Express, served via Node Forever and NodeMon
14. What’s available while programming in Node?
• Lambda Function Handler
• Event Object
• Context Object
• ImageMagick available (Node.js)
• Logging & Monitoring (CloudWatch & CloudTrail)
• Exceptions
• Ephemeral I/O operations while function is executing
• Awesome! But the worst part:
• Node v0.10.36
• http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html
15. Code via the online code editor, or in your own IDE
• Let’s build a simple API that goes out to Github, and get’s a user object.
• Here’s our Lambda function:
16. Testing your Lambda function locally
• Easy to test your logic locally
• Easy to turn this into an express module that you can then use
to serve via an api somewhere else so you’re not bound to
Lambda if you decide to move at a later date.
17. Upload your code
• Compress the items in you source
folder.
• Upload the zip file directly to your
Lamda function via the console.
• Better if you use a task manager like
Gulp that will zip the file and upload it
for you via the AWS SDK publish API
• I prefer to use Gulp to zip my code, then
upload to S3 so I have artifact
versioning as well. I wrote another
Lambda functions that listens for upload
events on that bucket then imports the
code into my Lambda function.
18. Now let’s create an API in API Gateway
that integrates with our Lambda Function
19. Let’s setup our API to have a query string parameter
called ‘name’ that we pass into our Lambda Function
20. Next we map the data retrieved byApi Gateway
to the Event object passed into our Lambda Function
via the mapping template
21. Let’s deploy our Api to a stage called ‘prod’!
Staging allows you to create various versions of your
Api for testing, different clients, etc…
23. Here’s my Lambda function that
deploys other Lambda Functions
• Event source for this Lambda function in my S3 bucket where I upload my
Lambda zip files
• Uses the AWS SDK which is already loaded into memory by AWS.
24. Pricing Details
• No hourly, daily, monthly minimums
• No per device fees
• Compute time is purchased in 100ms increments
• 1M requests and 400,000 GB-s of compute every month for free
• You only pay when you’re doing work, not when you’re idle.
Requests
You are charged for the total number of requests across all your functions. Lambda counts a request
each time it starts executing in response to an event notification or invoke call, including test invokes
from the console.
• First 1 million requests per month are free
• $0.20 per 1 million requests thereafter ($0.0000002 per request)
Duration
Duration is calculated from the time your code begins executing until it returns or otherwise terminates,
rounded up to the nearest 100ms. The price depends on the amount of memory you allocate to your
function. You are charged $0.00001667 for every GB-second used.
Free Tier
The Lambda free tier includes 1M free requests per month and 400,000 GB-seconds of compute time
per month. The memory size you choose for your Lambda functions determines how long they can run in
the free tier. The Lambda free tier does not automatically expire at the end of your 12 month AWS Free
Tier term, but is available to both existing and new AWS customers indefinitely.
25. Monthly compute charges
The monthly compute price is $0.00001667 per
GB-s and the free tier provides 400,000 GB-s.
Total compute (seconds) = 30M * (0.2sec) =
6,000,000 seconds
Total compute (GB-s) = 6,000,000 * 128MB/1024
= 750,000 GB-s
Total Compute – Free tier compute = Monthly
billable compute seconds
750,000 GB-s – 400,000 free tier GB-s = 350,000
GB-s
Monthly compute charges = 350,000 *
$0.00001667 = $5.83
Monthly request charges
The monthly request price is $0.20
per 1 million requests and the free
tier provides 1M requests per month.
Total requests – Free tier request =
Monthly billable requests
30M requests – 1M free tier requests
= 29M Monthly billable requests
Monthly request charges = 29M *
$0.2/M = $5.80
Total compute charges
Total charges = Compute charges
+ Request charges = $5.83 + $5.80
= $11.63 per month
Pricing Example
If you allocated 128MB of memory to your function, executed it 30 million times in one month,
and it ran for 200ms each time, your charges would be calculated as follows:
26. What should you do next?
• Take just one API and turn it into a Lambda function.
• Build a mobile app using the Mobile SDK using Lambda as
your backend
28. • Great example of scaling a total of 57 lines of code including error handling to
16 million posts a day.
• Scrub to 26:00 to see the Zillow presentation
https://www.youtube.com/watch?v=ygHGPnAd0Uo
32. Calling Lambda Functions (extra)
Call from mobile or web apps
- Wait for a response or send an event and continue
- AWS SDK, AWS Mobile, Rest API, CLI
Send events from S3, or SNS
- One event per Lambda invocation, 3 attempts
Process DynamoDB changes records as events:
- Ordered model with multiple records per event
- Unlimited retries (until data expires) or determined by your code
33. Writing Lambda Functions (extra)
• AWS SDK by default
• Imagemagick for NodeJS Lambdas
• Lambda handles the inbound traffic (no web server)
• Scheduled Functions! (uses cron syntax)
• Allows versioning via a publish API so you can create immutable contracts with your
clients, and you can version via S3
• (Versioning happens via the ARN format FunctionName:ARN or FunctionName:
$LATEST)
• You can alias the ARN, allows for each rollbacks by quickly associating an alias with a
different ARN. You can also version via the API in API Gateway by assigning the
resource to a different Lambda Function.
Stateless
• Use s3, DynamoDB, or any internet based storage to persist data.
• Call other API’s from your Lambda function, or other Lambdas.
• Uses processes, threads, sockets, i/o for file writing (ephemeral) etc…
• You can add your own libraries, no restrictions
Send events from S3, or SNS
- One event per Lambda invocation, 3 attempts
Process DynamoDB changes records as events:
- Ordered model with multiple records per event
- Unlimited retries (until data expires) or determined by your code