Migrating from a hosted environment to AWS is a good opportunity to streamline deployment and site operations. This session shows how FillZ used AWS OpsWorks with other tools to automate site operations and deliver a highly available site that is used by large numbers of customers. Through code and examples, this session shows you how to automate deployments across an entire fleet, configure a patching strategy, use common tools to create useful alarms and monitor system performance, and employ security best-practices in AWS.

1. November 13, 2014 | Las Vegas, NV
Jonathan Quail/ Cliff McCollum, FillZ.com

5. Story background…

7. https://www.flickr.com/photos/stevecorey/12844122413

9. Debian
CentOS
Amazon Linux
https://www.flickr.com/photos/bdesham/2432400623

12. https://www.flickr.com/photos/beigephotos/5334612
https://www.flickr.com/photos/colorhome/4033895230

17. Monitoring
Provisioning
Logging
Deployment
Modeling
Configuration management
Auto Scaling
Access management
Resource management

20. github.com/aws/opsworks-cookbookssupermarket.getchef.com

22. Hot tip!

23. Hot tip!

24. •Application servers
•Monitoring servers
•Graphite/dashboards
•Splunkcluster

32. –fillz-app::setup, fillz-app::configure, fillz-app::deploy
–fillz-base::setup, fillz-base::configure
Setup
Configure
Deploy
Undeploy
Shutdown

36. $easy_deploy.pydeploy –-application=myappinstances–-hosts=host1,host2
-–stack-name=my-test-stack--comment=“Deploying myappto host1/host2”
$easy_deploy.pydeploy –-application=myappall–-exclude-hosts=host1,host2
-–layer-name=backend-servers-–stack-name=my-test-stack
--comment=“Deploying myappto rest of the stack”
$easy_deploy.pydeploy –-application=myapprolling–-layer-name=apiservers
-–stack-name=my-api-stack--comment=“Deploying myappbehind ELB servers”

37. Code is available from the Githubrepo

39. By permission of Leigh Rubin and Creators Syndicate, Inc.

42. all_hosts={}
all_hostgroups={}
# Gets all (online only) instances in OpsWorksstack
all_instances= search(:node,'role:*')

43. all_instances.eachdo|instance|
node_hostgroups={}
# add a hostgroupfor each layer the instance is in
instance['opsworks']['layers'].each do|layer_id, layer_info|
node_hostgroups[layer_id] = layer_info['name']
end
# add a hostgroupfor each availability zone
node_hostgroups[instance['availability_zone']]=
instance['availability_zone']
all_hosts[instance['hostname']] ={
:hostgroups=> node_hostgroups,
:private_ip=> instance['private_ip']
}
all_hostgroups=all_hostgroups.merge(node_hostgroups)
end

44. template"/etc/nagios/conf.d/hostgroups.cfg"do
source'hostgroups.cfg.erb'
owner'nagios'
group'nagios'
mode00640
variables(
:hostgroups=> all_hostgroups
)
notifies:reload,'service[nagios]'
backup0
end

45. template"/etc/nagios/conf.d/hosts.cfg"do
source'hosts.cfg.erb'
owner'nagios'
group'nagios'
mode00640
variables(
:hosts=> all_hosts
)
notifies :reload,'service[nagios]'
backup0
end

52. Hot tip!

53. <infomercial>
</infomercial>
Wait…Remember that deployment tool? If you call in the next 20 minutes…it does patching too.

54. No “4 easy payments of $29.95” this part is open source too

55. $easy_deploy.pyupdate --amazon-linux-release=2014.09 --allow-reboots
instances–-hosts=host1,host2 -–stack-name=my-test-stack
--comment=“Upgrading host1/host2 to 2014.09 release”
$easy_deploy.pyupdate all–-exclude-hosts=host1
-–layer-name=backend-servers-–stack-name=my-test-stack
--comment=“Patching layer for ++opensslvulnerability”
$easy_deploy.pyupdate --allow-reboots rolling
-–layer-name=apiservers–-stack-name=my-api-stack
--comment=“Patching apibehind ELB servers and reboot for kernel”

58. “<stack><name>”

59. “bastion-devfillz-graphite” “bastion-devlocalhost:8000:fillz-graphite:80”

62. https://www.flickr.com/photos/wfryer/716965391

66. /var/log/aws/opsworks/opsworks-agent.log/var/log/aws/opsworks/opsworks-agent.keep_alive.log/var/log/aws/opsworks/opsworks-agent.process_command.loghttp://blogs.aws.amazon.com/application- management/post/TxTX72HFKVS9W9/Using-Amazon-CloudWatch- Logs-with-AWS-OpsWorks

68. BEFORE
AFTER
Time to deploya new server
2 days + 2 hours
25 minutes
Time to deploy anew software release
2 hours
2 minutes
After-hours alerts
6 / week
< 1 / week

69. BEFORE
AFTER
Scheduleddowntime per month
5 hours
0 minutes
Software deploymentrate
2 / month
2 / week

71. http://github.com/aws/opsworks-cookbookshttp://supermarket.getchef.comhttp://blogs.aws.amazon.com/application- management/post/TxZX2UI4YSBW3T/Customizing-AWS-OpsWorks-with-Attributeshttp://blogs.aws.amazon.com/application- management/post/TxTX72HFKVS9W9/Using-Amazon-CloudWatch-Logs-with-AWS- OpsWorkshttps://github.com/awslabs/reinvent2014-scalable-site-management

72. http://bit.ly/awsevals