The document discusses automating operations workloads using AWS services. It describes the workload of an IT infrastructure team, challenges they face, and best practices for designing and configuring automated workflows. Examples of automated disk space management and backup management workflows are provided, and a demo of disk space monitoring is given. The presenters seek questions and suggestions for additional automated workflows.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Automating Operations Workload
Shashi Prabhakar, Solutions Architect
Alessandro Martini, Cloud Support Engineer

2. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Who we are
Shashi Prabhakar, Solutions Architect
Alessandro Martini, Cloud Support Engineer

3. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Agenda
• Workload of an IT Infrastructure Team
• Challenges of IT Infrastructure Team
• Best Practices of Workflow
• Examples
• Demo

4. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Workload of an IT Infrastructure Team

5. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Many different hats
• Scope:
• Log Management
• Backup Management
• System Performance
• Monitoring
• Anti-Virus
• Security Scans

6. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Different Types of Workloads
• Periodically:
• Daily
• Weekly
• Monthly
• Yearly

7. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Different Types of Workloads
• Environment Interaction:
• Infrastructure
• Operating System
• Proactive Vs Reactive

8. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges of IT Infrastructure Team

9. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges Today
• Multiple AWS Consoles

10. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges Today
• Multiple AWS Services

11. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges Today
• No Unified workflow

12. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges Today
• Best Practices not defined

13. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices
Workflow Design and Configuration

14. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
Answer the following questions:
• How does my workflow start?
• Schedule, API event, Manual trigger
• Does my workflow need to interact with EC2 instances at the OS level?
• Infrastructure workflow, OS workflow, Infrastructure + OS workflow
• What is the output?
• An AMI, a true/false value, etc..
• How does a successful output look like?
• An AMI can be restored, all checks return ‘true’, etc..

15. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
AWS
Lambda
AWS
Config
Trigger Action
AWS Step
Functions
Output Compliance
Run Command
Automation
Amazon
CloudWatch
State Manager

16. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
Trigger
Amazon
CloudWatch
Cloudwatch Events
• Triggers an action based on an API event or cron job
• Can work across AWS accounts
• Supports Amazon EC2 Systems Manager Run
Command, AWS Lambda, Step Functions and other
services as Target
State Manager

17. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
AWS
Lambda
Trigger Action
AWS Step
Functions
Amazon
CloudWatch
Amazon EC2 Systems
Manager
• 7 different components. We
are using 3 - State Manager
(trigger), Run Command and
Automation (action)
• State Manager can trigger
Run Command
• Run Command is the only
service that can access OS
information
State Manager
Run Command
Automation

18. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
AWS
Lambda
Action
AWS Step
Functions
AWS Step Functions
• Orchestrate multiple
Lambda functions
• Code complex logic
(branching, retry logic)
AWS Lambda
• Runs any AWS script, at
scale
• No server needed
Run Command
Automation

19. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow design
AWS
Config
Output Compliance
AWS Config
• AWS Config rule are created to show Compliance
and Non-Compliance of AWS resources
• An AWS Account is a resource itself
• Custom rule are backed by AWS Lambda to
implement your custom logic

20. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow configuration
Use CloudFormation to create, manage and maintain a workflow
• One single console for configuration changes
• Consistent deployment
• Workflow is region and account agnostic
• You don’t need to know everything about the implementation to enable a
workflow

21. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices – Workflow configuration
Use CloudFormation template parameters as high-level input for your workflow:
• Schedule
• Workflow specific settings (i.e. backup retention, alarm thresholds)
These parameters must be understandable for anyone that is familiar with the
workflow, but not necessarily with the implementation.
Examples:
• Ask for the backup retention days, not for the AMI retention days
• Ask for the disk space utilization threshold, not for the AWS Config custom
rule parameter for the compliance check.

22. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Best Practices

23. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Example Workflow – OS – Disk Space Management
• How does my workflow start?
• Schedule
• Does my workflow need to interact with EC2 instances at the OS level?
• Yes (OS workflow)
• What is the output?
• An SSM custom inventory value per partition, with % usage.
• How does a successful output look like?
• % usage is < threshold

24. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Example Workflow – OS – Disk Space Management
AWS
Config
Trigger Action Output Compliance
Run CommandState Manager

25. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Disk Space Management
Compliance checks:
- required-disk-space:
Checks usage % of each
disk partition of an EC2
instance in an
environment
Send Inventory Request
Notify IT Infrastructure Team
about non-compliance
Trigger Action Output
Compliance
State Manager Run Command Custom
Inventory
AWS
Config
Amazon
SNS
AWS
Lambda

26. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Output

27. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Demo
Disk Space Management

28. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Example Workflow – Infrastructure – Backup Management
• How does my workflow start?
• Schedule
• Does my workflow need to interact with EC2 instances at the OS level?
• Both OS and Infrastructure (Infrastructure + OS workflow)
• What is the output?
• An AMI of an EC2 instance
• How does a successful output look like?
• My EC2 instance has an AMI. An AMI has the following properties: tested
(OS as well), deleted if older than X days, or if it can’t be restored

29. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Example Workflow – Infrastructure – Backup Management
Amazon
CloudWatch
AWS
Config
Trigger Action Output Compliance
AWS
Lambda
Run Command
Automation

30. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Example - Backups Management
AWS
Lambda
AWS
Config
Compliance checks:
- required-volume-
backups: Check
existence of backups for
each EBS volume of an
EC2 instance in an
environment
- volume-backups-
retention: Backups are
not older than X days Amazon
SNS
Create Daily Backup
Validate Backups
AWS
Lambda
Delete Old Backups
Notify IT Infrastructure Team
about non-compliance
Trigger Action Output
Compliance
Automation
AWS
Lambda

31. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Demo
Backup Management

32. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What’s Next
• More Automated Workflows (suggestions?)
• Blog Posts
• Remediation

33. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Questions?

34. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Shashi Prabhakar, Solutions Architect
Alessandro Martini, Support Engineer
Thank you!
Automating Operations Workload