Learning Objectives: - Learn how to use AWS OpsWorks, AWS CodeDeploy, and AWS CodePipeline to build a reliable and consistent development pipeline - Understand about continous integration and delivery for Infrastructure as Code - Learn how to get started with these services.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Leo Zhadanovsky
Principal Solutions Architect
Amazon Web Services
September 18, 2017
Configuration Management in the
Cloud
Automation without undifferentiated heavy lifting

2. DevOps – culture and process
Kaizen
Improvement
Activities that continuously improve all
functions and involve all employees.
By improving standardized activities and
processes, we aim to eliminate waste.

3. Traditional on-premises configuration management
• Long-lived VMs or physical
hosts
• Static DNS
• Heavyweight configuration
management on a static
server
• Challenges distributing
keys/secrets

4. Traditional on-premises configuration management
• Configuration management
libraries are complex and
take on a life of their own
• Require specialist
developers
• Require unit tests and
continuous integration—
just like your app code
• Adds thousands of LOC of
complexity just to manage
machines

5. Undifferentiated heavy lifting moves up the stack
• Undifferentiated heavy
lifting: necessary but
repetitive work to support
your application
• Originally: Data centers,
power, network, storage
• Now: Complex configuration
management, monitoring,
logging

6. Cloud is different: Auto Scaling
• Auto Scaling allows you to
meet your workload with
fewer hosts during low
periods
• Hosts are shorter-lived
• Long convergence times
typical of traditional CM
no longer acceptable
• Patching and long-term
maintenance simplified
by replacing base image

7. Deployment and Management Services
AWS
CloudFormation
AWS
CodeDeploy
AWS
OpsWorks
AWS
Elastic Beanstalk

8. Deployment and Management Services
CloudFormation
OpsWorks
CodeDeploy
Elastic Beanstalk

9. AWS CloudFormation
• Declare all components of
your application in a JSON
template
• Treat as your architecture’s
“concrete”
• Virtual Private Cloud
• Subnets
• Security Groups
• Auto Scaling Groups
• Load Balancers

10. AWS CloudFormation
• Modify your resources in
the template and resubmit
to change your
infrastructure
• Keeps security groups,
etc. safely managed in
config files

11. AWS CloudFormation
• Package and share your
entire infrastructure as a
single file
• Replicate your environment
across multiple Regions
• Share your application’s
complete environment with
others
Application
“Resources” : {
“ELB”,
“AutoScaling”,
“RDS”
},
"Mappings": {
“AMIWebApp": {
“us-east-1": { "64": "ami-x" },
"us-west-1": { "64": "ami-y" },
"us-west-2": { "64": "ami-z" }
}
}

12. cfn-init
• Agent running on an EC2
instance, typically invoked on
boot as part of template
creation
• Reads instance metadata
stored in the template
• Installs files, runs scripts,
creates users, starts services
AWS CloudFormation helpers
cfn-hup
• Runs on an EC2 instance as a
service
• Checks for updates to the
CloudFormation template and
can run any shell script
• Typically used to call cfn-init to
update configuration files or
other settings

13. Use AWS::CloudFormation::Init
"install_application" : {
"commands" : {
"01_get_chef" : {}, ...,
"02_configure_node_run_list" : {
"command" : “chef-client –r ‘recipe[application]’",
"cwd" : "/var/chef/chef-repo",
"env" : { "HOME" : "/var/chef" }
}
}
}

14. AWS CodeDeploy
• Use CodeDeploy to deploy
applications to Linux or
Windows instances
• Similar semantics as
package management tools
• Write scripts for lifecycle
hooks:
• ApplicationStop
• BeforeInstall
• AfterInstall
• ApplicationStart
• ValidateService

15. AWS CodeDeploy
• Use deployment strategies
to implement rolling
deployments
• Safely deploy new versions
of your code
• Safely roll back by
deploying previous versions
• Decouple your
infrastructure from
deployment automation

16. Amazon Elastic Beanstalk
Fast & simple
to begin
Developer
productivity
Impossible
to outgrow
Complete
resource control

17. • Capacity provisioning,
load balancing, auto
scaling, and health
monitoring is handled for
you
• PHP, Python, Java, Ruby,
Node.js, .NET, Go, Docker
Amazon Elastic Beanstalk
Amazon
CloudWatch
AWS
CodePipeline
AWS Toolkit for Eclipse
AWS CLI
AWS Toolkit for Visual
Studio
Amazon
ElastiCache
Amazon RDS
Auto Scaling Group
EC2 Instance EC2 Instance EC2 Instance
Region
Elastic Load
Balancing

18. AWS OpsWorks
Dynamic configuration and orchestration
Automatic instance scaling and auto
healing
Build each new instance to your
specification
Change instance configuration
in response to system events

19. AWS OpsWorks

21. What is Chef Automate?
• Refer to your infrastructure as code (cookbooks & recipes)
• Consistently install, configure, manage, deploy and scale
applications
• Align resources with specific policies
• Save time by automating manual tasks

22. How does it work?
• Simple client-server
architecture
• Connecting resources to a
Chef server
• Resources pull
configuration updates from
the Chef server Config A Config B

23. How can you set this up?
1. Setup the Chef server with cookbooks, recipes roles.
2. Install the Chef client on the instance (or server).
3. Register the instance with the Chef server as a Chef node.
4. Assign node with a role (e.g. web server, app server, db server).
5. The Chef client asks the Chef server for a set of recipes (instructions).
6. The Chef server determines the applicable recipes (by role).
7. The Chef client applies the recipes on the node by doing a “Chef run”.
8. The Chef client pulls the Chef server every 30 minutes.

24. How does it look like?
• The Chef client pulls
configuration updates from the
Chef server every 30 minutes.
• The Chef client will only make
configuration changes when
the node is out of spec.
• The Chef client can react to
changes using by using Chef
search.

25. Chef recipe example – configure Apache
# Install Apache and start the service.
httpd_service ‘default' do
listen_ports ['81', '82']
threadlimit '4096'
action [:create, :start]
end
# Add the site configuration.
httpd_config ‘default' do
instance ‘default'
source ‘mysite.conf.erb'
notifies :restart, 'httpd_service[default]'
end
.....

26. Chef recipe example – configure Apache
# Create the document root directory.
directory '/var/www/default/public_html' do
recursive true
end
# Write the home page.
file '/var/www/default/public_html/index.html' do
content '<html>This is a placeholder</html>'
mode '0644'
owner 'web_admin'
group 'web_admin'
end
.....

27. Chef recipe example – configure PHP
# Install the mod_php5 Apache module.
httpd_module 'php5' do
instance ‘default'
end
# Install php5-mysql.
package 'php5-mysql' do
action :install
notifies :restart, 'httpd_service[default]'
end

28. Get visibility into the state of your nodes
Visibility – A view into convergence, compliance, cookbooks, recipes and more.

29. Not only a Configuration Management tool
Workflow – A continuous delivery pipeline of infrastructure and applications.

30. Not only a Configuration Management tool
Compliance - Discovery and analysis of compliance risks across environments

31. AWS OpsWorks
for Chef Automate

32. What is AWS OpsWorks for Chef Automate?
The place you go to for configuration management on AWS
Offers a fully managed Chef Automate server
OpsWorks

33. How can I create an AWS managed Chef server?
Easy to get started, get a Chef Automate server in 10 minutes.

34. What else can I set up?
Setup a weekly maintenance window
• Automatic security updates
• Automatic Chef version upgrades

35. What else can I set up?
Setup a daily/weekly backup schedule

36. What else is left for me to do?
Nothing, this is a fully managed configuration management
service:
• Automatic backups
• Automatic security updates
• Automatic Chef software updates
You can focus on writing cookbooks and recipes that meet
your needs.

37. What other benefits do I get from the service?
• Automatic instance to Chef server registration
• Secure and easy scaling using Auto Scaling Groups
• No separate license fees, only pay for what you use
• Best practices, AWS support and guidance

38. Where does it come in the tool chain?
• Bootstrap instances with the right configuration
• Update the configuration of running instances
• Assure instances comply with a pre-defined policy
Can be a part of your Continues Integration and Continues
Delivery pipeline

39. Amazon EC2 Systems Manager
Thin automation bootstrap layer
• Auto-domain join when launching Windows instances
• Supports joining in AWS Directory Service through Simple AD and AD
Connector
• Installation of PowerShell modules
• Installation of MSI packages
• Configure CloudWatch metrics and logs
Complementary to PowerShell DSC/Chef, etc.
• Use Systems Manager to bootstrap
• Optionally, hand over to other tools for more in-depth
configuration

40. Parameter Store
• Encrypt sensitive information using your own KMS keys
• Reference your parameters in Run Command, State Manager,
or Automation service
• Use with IAM to manage access in a granular fashion
• Eliminate ongoing maintenance challenge of critical enterprise
assets
Centralized management of IT assets such as passwords
and connection strings

41. Infrastructure as code
• The ability to completely
specify an environment is a
key advantage of cloud
• No extra environment
setup required
• Allows operations resources
to scale by sharing
complete environments

42. DevOps & Security
• Traditional security work is
hard to scale
• Instead, provide reusable
security deliverables by
creating common template
components and machine
images
• Codify and enforce
compliance requirements as
custom AWS Config Rules
• Amazon Inspector

43. Toward a toolbox approach
• Use CloudFormation and
other tools to create
reusable components that
application teams can
leverage
• Work on code, not
servers—even virtual ones
• Leverage lighter-weight and
managed tools to eliminate
complexity

44. Organize by layers & environments
Layers of stacks
Environments
Frontend
Services
• Consumer Website, Seller Website,
Mobile Backend
Backend
Services
• Search, Payments, Reviews,
Recommendations
Shared
Services
• CRM DBs, Common Monitoring
/Alarms, Subnets, Security Groups
Base
Network
• VPCs, Internet Gateways, VPNs,
NATs
Identity • IAM Users, Groups, Roles

45. Apply service-oriented architecture
Food Catalog
website
Ordering website
Customer DB
service
Inventory service
Recommendations
service
Analytics service
Fulfillment
service
Payment
service

46. Nested stacks for reusability & specialization
Application1
“Resources” : {
“ELB”,
“AutoScaling”,
“RDS”
}
Application2
“Resources” : {
“ELB”,
“AutoScaling”
}
ELB_AND_AS
“Resources” : {
“ELB”,
“AutoScaling”,
“Networking”
}
Application1
“Resources” : {
“NestedStack”,
“Networking”
}
Application2
“Resources” : {
“NestedStack”,
“DynamoDB”
}

47. Next Steps
• Amazon Inspector
• AWS Lambda
• AWS WAF

48. Thank you