Defining infrastructure resource policies in an organized manner can help your company better manage its infrastructure resources. This session familiarizes you with using AWS Lambda to process data and provide control logic for your infrastructure. You can use Amazon CloudWatch Events to monitor infrastructure resources in real-time, and you can use AWS Lambda to react to events based on a set of rules. We demonstrate how you can build a rules engine for creating, monitoring, and managing policies. This is all done using Alexa and Alexa Skills Kit.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Randall Hunt, Developer Evangelist, AWS
Using AWS Lambda to Build Automated
Control Systems for Your AWS
Infrastructure
December 1, 2016
SVR401

2. Welcome to re:Invent!
Randall Hunt - @jrhunt
 Developer Evangelist at AWS
 randhunt@amazon.com
 Formerly of SpaceX, NASA

3. Agenda
• Brief overview of AWS Lambda
• Why automate?
• Why Lambda for automation and control systems?
• Event-driven policy enforcement
• Lambda as an infrastructure control plane
• Best practices

4. Operations and management Scaling
Provisioning and utilization Responsible for availability
and fault tolerance
Owning servers means dealing with ...

5. Serverless compute: AWS Lambda
COMPUTE
SERVICE
EVENT-
DRIVEN
Run code
without
managing
servers
Code runs only
when it needs to
run

6. Code is all you need
(Native: Java/Python/NodeJS)
Event-driven scaling
Never pay for idle servers Availability and fault tolerance built in
Going serverless with AWS Lambda

7. Things to remember: Lambda function
 Memory = “Power level”
• Higher levels offer more memory and more CPU power
 Functions don’t have a notion of state
• Use Amazon DynamoDB, Amazon S3, or Amazon ElastiCache
• Wrap your config in a function and call it from your published code
 Use the right access control for downstream services
• IAM roles and permissions for AWS services
• VPC for private endpoints

8. Why automate?

9. Benefits of automation done correctly
Increased Agility Improved Quality
Improved Innovation
Increased Consistency
Reduced Expenses and
Labor Costs
Reduced Human Errors

10. Dangers of incorrect automation
Code Maintenance Issues Bad Permissions Control Auditing Nightmares
Everyone Blames You!Accountability

11. Why is automation key?
Amazon EC2 Amazon ECS Amazon CloudWatch AWS CloudTrail AWS IAM

12. What sorts of things can we automate?
Just about anything!

13. Amazon
API Gateway
AWS
IoT
Amazon
Alexa
Automatic Infrastructure Cleanup Allow other endpoints and AWS services to
trigger infrastructure change
Create copies/snapshots of data in response
to events
Integrate / think outside the box
Amazon EC2 Amazon ECS Amazon ECR
Amazon RDS Amazon
DynamoDB
Instant Core Infrastructure Changes
Amazon
VPC
Amazon
Route 53
AWS
CloudFormation

14. Why Lambda for automation?

15. Lambda as a building block for automaton
Because Lambda is event-driven, it offers a very
powerful framework for automated infrastructure
control planes and policy engines.
AWS Lambda
Near real-time reactionEvent

16. Benefits for automation with AWS Lambda
Single knob configuration Easy to integrate Logging and auditability
Rapid delivery and versioning Get started fast

17. Event-driven automation as a building block
Being proactive rather than reactive to changes in your
infrastructure is key. However, event-driven automation can
be part of the solution. We can remove human errors when
we need to be reactive through use of automation.

18. Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
AWS
CloudFormation
AWS
CloudTrail
Amazon
CloudWatch
Logs
Amazon
SNS
Amazon
SES
Amazon
API Gateway
Amazon
Cognito
AWS
IoT
Cron events
Trigger an event based on Data Store changes Interact with other endpoints and AWS services
to trigger infrastructure change
Respond to custom events as they happen
near real-time
Respond to messaging or scheduled
events in your infrastructure
Event sources

19. Event-driven automation
EC2 Instance
“Event”
Manual Event
Data Streams / Logs
Lambda.Invoke

20. Event-driven policy enforcement

21. Policy enforcement
What do you mean by policy enforcement?
You have the freedom to define what
policy enforcement means to you!

22. Policy enforcement
AWS Config Rules Amazon
CloudWatch
AWS
CloudTrail
Lambda Policy Engine

23. AWS Config
Resource Inventory

24. AWS Config
Configuration History

25. AWS Config
Config Rules

26. VPC policy enforcement
Lambda Policy Engine
• Allocate Address
• Associate Address
instance instance instance
instance instance instance
instance instance instance
Validate Security Group
instance instance instance

27. Identity and access management enforcement
Lambda Policy Engine
Validate Role Path
AWS IAM
CreateRole
Verify No Role Elevation
Verify No Resource Level Elevation
Allow Role
Delete Roles

28. Tagging enforcement
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstances
Validate Team Tag
Validate Billing Group Tag
Validate Environment Stage Tag
instance instance instance
StopInstances
SNS Notification
instance instance instance

29. Demo!
AWS Config compliance checks

30. Infrastructure control plane

31. Infrastructure control plane
Lambda gives you the freedom to
design the automation you need for
your infrastructure.

32. Infrastructure automation
Amazon CloudWatch Events
Auto Scaling Event AWS API Call EC2 Instance State Change

33. Automatic security group placement
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstances
Check VPC
Validate Security Group
Remove SG
Add Correct SG
instance instance instance
Do Nothing

34. Create/Update Amazon Route 53 records from tag
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstances
Do Nothing
instance instance instance
Check VPC
Update DNS instance
Get CNAME Tag
Add DNS Record
instance instance

35. Demo!
Automated Route 53 DNS updates on instance creation

36. CloudFormation Bootstrapping

37. Bootstrapping With Lambda and Await/Notify
• Need to load a schema into RDS? Lambda function.
• Need to load data into RDS? Lambda function.
• Need to load data into DynamoDB? Lambda function.
• Need to perform a complex bootstrapping action?
Lambda function.
• Need to depend on an external action? Lambda function
+ API Gateway.
• For everything else there’s EC2 dependson/await/notify.

38. Best practices

39. Best practices
Log Everything
Bite-Sized Code
Report Failures
Version Lambda Functions
Use Source Control
Iterate

40. Recap
• Event-Driven Response to Policy Management and Infrastructure Events.
• AWS Config Rules backed by Lambda to visualize compliant versus non-compliant
infrastructure.
• Think outside the box. Number of available CloudWatch Events API Call triggers is
large.
• Don’t forget the ability to schedule AWS Config rule validation, as well as Scheduled
CloudWatch Event Triggers to Lambda.
• CloudWatch Events can point to multiple Targets (Not just Lambda functions).

41. Thanks for your attention!
Follow @jrhunt

42. Remember to complete
your evaluations!