Puppet Camp Melbourne 2015

1. ©2015,
Amazon
Web
Services,
Inc.
or
its
affiliates.
All
rights
reserved
Shiva N
Solutions Architect, AWS
@shivamaan
AWS + Puppet = Dynamic Scale

2. Application Rate of Change Is Increasing
Source 2013 State of DevOps Report
Puppet Labs, IT Revolution Press
Top Performers
30x more frequent
code deployments
Of those with DevOps
implemented > 12 month:
More than 25% can
deploy on demand

3. ~11.6s
Mean time
between
deployments
(weekday)
~1,079
Max number of
deployments in a
single hour
~10,000
Mean number of
hosts
simultaneously
receiving a
deployment
~30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT AMAZON.COM

4. 75%
Reduction in
outages triggered
by software
deployments since
2006
90%
Reduction in
outage minutes
triggered by
software
deployments
Business Value of Frequent
Deployments
~0.001%
Software
deployments
cause an
outage

5. example.com – Online auction
AWS Sydney Region
Public subnet App subnet
HAProxy Vertx
Glassfish
Data subnet
Puppet
Stack
ELK
StackMgmt subnet
CloudFront S3 SES SNS
Route 53
Jenkins

6. Let’s build
[ ] OS
[ ] Patches
[ ] OS Configuration
[ ] VM Templates (Amazon Machine Image)
[ ] VPC
[ ] Networking (Subnets, ELBs)
[ ] Network Security (NACLs and SGs)
[ ] Software
[ ] Application (Code)
[ ] Services

7. Start with a Base Image
Machine Images can come from several sources..
- AWS Provided (Quick Start)
- AWS Marketplace
- Community AMIs
- Build your own AMIs
- Migrate On Premise Image
We need a server to
build an image to
build a server

8. Let’s build
[X] OS
[ ] Patches
[ ] OS Configuration
[ ] VM Templates (Amazon Machine Image)
[ ] VPC
[ ] Networking (Subnets, ELBs)
[ ] Network Security (NACLs and SGs)
[ ] Software
[ ] Application (Code)
[ ] Services

9. Layers to Golden Image
CI-­‐CD Pipeline
- git push puppetmanifests
- Packer/AMINator
- puppet apply
- Test image
- Register AMI, meta-­‐data in CMDB

10. Let’s build
[X] OS
[X] Patches
[X] OS Configuration
[X] VM Templates (Amazon Machine Image)
[ ] VPC
[ ] Networking (Subnets, ELBs)
[ ] Network Security (NACLs and SGs)
[ ] Software
[ ] Application (Code)
[ ] Services

11. Cloudformation

12. Let’s build
[X] OS
[X] Patches
[X] OS Configuration
[X] VM Templates (Amazon Machine Image)
[X] VPC
[X] Networking (Subnets, ELBs)
[X] Network Security (NACLs and SGs)
[ ] Software
[ ] Application (Code)
[ ] Services

13. Build Puppet Stack
- EC2 Metadata service
$ curl http://169.254.169.254/latest/user-­data
hiera_role=PuppetMaster
aws s3 cp s3://puppetmanifests /etc/puppet/modules
- Self awareness by hiera magic
- Self manifestation with puppet apply

14. Build Software Stack
- CloudFormation
- RDS
- Elasticache
- S3
- CloudFront
- Route 53
- ELB
- SNS
- Autoscaling groups
- PuppetMaster
- Build ELK Stack (PuppetForge)
- Build Glassfish layer (Custom types and providers)
- Build Vertx layer (Custom types and providers)
- Build HAProxy layer (PuppetForge)

15. Let’s build
[X] OS
[X] Patches
[X] OS Configuration
[X] VM Templates (Amazon Machine Image)
[X] VPC
[X] Networking (Subnets, ELBs)
[X] Network Security (NACLs and SGs)
[X] Software
[ ] Application (Code)
[ ] Services

16. Application Deployment
define glassfish::app (
$user = 'glassfish',
$action = 'deploy',
$bindaddr = '127.0.0.1',
$mgmtport = '8080',
$deploy_source = '${glassfish::real_instance_basedir/deployments/${name}.war',
) {
require glassfish
if ($action == 'deploy') {
app { "${name}":
ensure => present,
mgmtport => $mgmtport,
user => $user,
source => $deploy_source,
}
}
}

17. Let’s build
[X] OS
[X] Patches
[X] OS Configuration
[X] VM Templates (Amazon Machine Image)
[X] VPC
[X] Networking (Subnets, ELBs)
[X] Network Security (NACLs and SGs)
[X] Software
[X] Application (Code)
[X] Services

18. example.com – Online auction
AWS Sydney Region
Public subnet App subnet
HAProxy Vertx
Glassfish
Data subnet
Puppet
Stack
ELK
StackMgmt subnet
CloudFront S3 SES SNS
Route 53
Jenkins

19. Let’s run
[ ] Tune
[ ] Release
[ ] Orchestrate and Automate

20. Decision, Decisions…
- In place deployments (Puppet run)
- MCollective
- CodeDeploy
- Immutable deployments
- Blue-­‐Green deployment
- AMI baking

21. AWS CodeDeploy

22. Bake an AMI Puppet +
Sweet spot
To AMI or not to AMI
Start an instance
Configure the instance
Create an AMI from your
instance
Start new ones from the AMI
Launch an instance
Use metadata service to get
hiera roles
Puppet run performs post
launch config

23. CI Pipeline
• Vagrant
• Jenkins
– Minor release
• CodeDeploy release to environment
• Packer build Golden AMI* -­‐> Update DynamoDB -­‐>
Schedule ASG refresh
– Major release
• Packer build Golden AMI* -­‐> Update DynamoDB -­‐> Refresh
ASG
* Golden AMI – Glassfish + HAProxy + Vertx + Applications with all services turned off

24. Thank You