The document discusses using Packer and Puppet for cloud infrastructure provisioning, emphasizing the importance of creating preconfigured machine images for reliability and speed. It highlights how Packer can build identical images across platforms and suggests using Git for managing Puppet configurations and sensitive data through hiera-eyaml for encryption. Additionally, it outlines the process of integrating IAM roles for accessing sensitive keys from an S3 bucket during the build process.

1. Baking in the Cloud
with Packer and
Puppet
Alan
Parkinson
@alan_parkinso
n

3. Large jumps in demand, we
have to scale fast
Load Balancer Behave Pro App

4. Our problems with
provisioning on start-up

5. Reliability
Many single points of
failure
Require Puppet
Master redundancy
and datacentre
replication
Mirror dependant
software repositories

6. Provisioning Latency…
9 minutes for puppet to prepare a app server
Total 15 minute response time to a scaling request

7. Baking images like
Produce a
preconfigured
machine image
that is rolled out
to the autoscaling
groups

8. Packer is a tool for
creating identical
machine images
for multiple
platforms from a
single source
configuration
PACKER.IO A HASHICORP PROJECT

9. PACKER.IO A HASHICORP PROJECT
packer.json
{
"variables": {
"aws_access_key": "",
"aws_secret_key": ""
},
"builders": [{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "us-east-1",
"source_ami": "ami-9eaa1cf6",
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"ami_name": "packer-example {{timestamp}}"
}]
}

10. $ packer build -var 'aws_access_key=YOUR ACCESS KEY'
-var 'aws_secret_key=YOUR SECRET KEY'
example.json
==> amazon-ebs: amazon-ebs output will be in this color.
==> amazon-ebs: Creating temporary keypair for this instance...
==> amazon-ebs: Creating temporary security group for this instance...
==> amazon-ebs: Authorizing SSH access on the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Waiting for instance to become ready...
==> amazon-ebs: Connecting to the instance via SSH...
==> amazon-ebs: Stopping the source instance...
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: packer-example 1371856345
==> amazon-ebs: AMI: ami-19601070
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
==> amazon-ebs: Build finished.
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-19601070

11. Add some provisioners
{
"variables": {…},
"builders": […],
"provisioners": [
{
"type": "shell",
"script": "../common/install-puppet.sh"
},
…
{
"inline": [
"sudo apt-get purge --yes puppet",
"sudo apt-get autoremove --yes"
],
"type": "shell"
}
]
}

12. No need for a Puppet Master
"provisioners": [
…
{
"type": "puppet-masterless",
"hiera_config_path": "../puppet/hiera.yaml",
"manifest_file": "../puppet/manifests/default.pp",
"module_paths": [ "../puppet/modules" ]
},
…
]
Manifests, modules and hiera data can all be
stored in git and git submodules

13. How do we protect
sensitive configuration
data?

14. hiera-eyaml
backend for Hiera that provides per-value
encryption of sensitive data within yaml files
---
duo-security-skey: ENC[PKCS7,MIIBmQYJKoZIh……Anc=]
behave_pro:
logentries_api_key: ENC[PKCS7,MIIBmQYJKoZIh……uW8=]
application_secret: ENC[PKCS7, MIIBmQYJKoZIh……FRg==]
common.eyaml

15. hiera.yaml
---
:backends:
- eyaml
- yaml
:hierarchy:
- "%{environment}"
- common
:yaml:
:datadir: '/tmp/hieradata'
:eyaml:
:datadir: '/tmp/hieradata'
:pkcs7_private_key: /tmp/hierakeys/private_key.pkcs7.pem
:pkcs7_public_key: /tmp/hierakeys/public_key.pkcs7.pem
Note: Use a temporary folders or data will be baked into the final
image

16. Basic asymmetric encryption
(PKCS#7)
Private key decrypts data
Puppet Master or agent only needs this at runtime
$ eyaml encrypt -s 'hello there'
Public key encrypts data
Safe to distribute to developers and ops engineers
Git diff allows peer review without decrypting values

17. The eyaml keys are stored in a private S3
bucket with access controlled by a IAM
Policy
Distributing the keys
to the Bakery
Use a IAM Role in Packer to access the S3
bucket
"builders": [{
"type": "amazon-ebs",
…
"iam_instance_profile" : "puppet-provisioner",
…
}]

18. IAM Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1425244502000",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::puppet.behave.pro/*"
]
}
]
}

19. install-heira-key.sh
Download the keys to the EC2
Instance
sudo apt-get install --yes python-pip
sudo pip install s3cmd
s3cmd get s3://puppet.behave.pro/private_key.pkcs7.pem
/tmp/hierakeys/private_key.pkcs7.pem
s3cmd get s3://puppet.behave.pro/public_key.pkcs7.pem
/tmp/hierakeys/public_key.pkcs7.pem
"provisioners": [
{
"type": "shell",
"script": "../common/install-hiera-key.sh"
},
]
packer.json

20. Summary
If scaling fast or reliably are
important, bake images
Git makes a great alternative to
Puppet Master when baking
Secure data with hiera-eyaml
https://github.com/TomPoulton/hiera-eyaml

21. Questions