0
DEV TO DELIVERY WITH
PUPPET, VAGRANT AND AWS
SAM BASHTON, BASHTON LTD
ABOUT ME
Linux guy since Slackware, floppy disks and root + boot
Using Puppet since 2007
Run a company in Manchester, Nort...
TOOLS FOR THE DAY
WHAT IS THE POINT OF THIS
TALK?
WHAT WE HAVE
Dev
Integration
QA
Stage
Live
WHICH ENVIRONMENTS ARE
MANAGED BY PUPPET?
Dev
Integration
QA
Stage
Live
WHAT WE'RE AFTER
Confidence that everything will work correctly in
production
Consistency between environments
OPS AND DEVS COOPERATING
Previously:
Devs built stuff
Later, Ops came and built production infrastructure
This caused many...
OPSVELOPMENT
OPS AND DEVS WORKING
TOGETHER
Ops need to be involved in development planning process
Puppet modules and manifests should ...
DEVELOP ON PUPPET
PROVISIONED
ENVIRONMENTS
As early as possible, all dev should be done on systems
built from Puppet
Puppe...
VAGRANT
Builds virtual machines, optionally from Puppet manifests
Makes it easy to spin up short-lived dev instances
Quick...
A WORKFLOW
Development happens on Vagrant VM(s)
Deployment to all shared environments happens via
Jenkins
PUPPET CONFIG
There should be only one set of Puppet
manifests/modules
Tested deployed and merged through software test
en...
ONE SET OF MANIFESTS,
MANY ENVIRONMENTS
Different environments need different config
Resource locations
Settings
DEALING WITH DIFFERENT
ENVIRONMENTS
Hiera
Removes the need for ugly if/else blocks
Put anything that differers by environm...
HIERA.YAML
:irrh:
heacy
-%evrnet
{niomn}
-cmo
omn
VAGRANTFILE
VgatcniueVGATIEAIVRIN d |ofg
arn.ofgr(ARNFL_P_ESO) o cni|
cni.mbx="ets4lc
ofgv.o
cno6-x"
cni.mhsnm ="uptofeape...
TO DEVELOP:
Start of the day, dev runs v g a t u and gets the latest
arn p
environment
Code/objects sit in a shared vagran...
VAGRANT PROVISIONERS
Avoid VirtualBox wherever possible
Slow, prone to taking down host machine
On Linux, vagrant-lxc is s...
VAGRANT AND AWS
Use Vagrant to bring up machines in AWS using
v g a t a splugin
arn-w
Makes it easy to share work in progr...
QA/STAGING
ENVIRONMENTS IN AWS
Merge to appropriate branch in git
Jenkins takes over
ADVANTAGE OF AWS
Great thing about AWS - we don't need to run our test
environments all the time
Have the environments onl...
TESTING VS LIVE
Use the money saved to build better environments
Minimise differences between testing and live
In particul...
SPEEDING UP THE PROCESS
Some resources, in particular DBs can be slow to provision
(30 mins plus)
Could just run 24/7
One ...
PILOT LIGHT PROVISIONING
Tiers built using autoscaling groups
Minimum instance count is 0
Jenkins sets desired capacity ap...
CONCLUSIONS
Infrastructure development should run in parallel to
software dev
This means devs + ops must co-operate
Minimi...
QUESTIONS? COMMENTS?
Sam Bashton
sam@bashton.com
Twitter: @bashtoni
(Psst.. http://www.bashton.com/jobs/ )
REFERENCES, LINKS
Vagrant
vagrant-lxc
hiera-gpg
Masterless Puppet + AWS
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Upcoming SlideShare
Loading in...5
×

Dev to Delivery with Puppet, Vagrant and AWS

2,668

Published on

"Dev to Delivery with Puppet, Vagrant and AWS" by Sam Bashton of Bashton Ltd. at Puppet Camp London 2013. Find the video here: http://puppetlabs.com/community/puppet-camp

Transcript of "Dev to Delivery with Puppet, Vagrant and AWS"

  1. 1. DEV TO DELIVERY WITH PUPPET, VAGRANT AND AWS SAM BASHTON, BASHTON LTD
  2. 2. ABOUT ME Linux guy since Slackware, floppy disks and root + boot Using Puppet since 2007 Run a company in Manchester, North West England We provide outsourced ops for other companies
  3. 3. TOOLS FOR THE DAY
  4. 4. WHAT IS THE POINT OF THIS TALK?
  5. 5. WHAT WE HAVE Dev Integration QA Stage Live
  6. 6. WHICH ENVIRONMENTS ARE MANAGED BY PUPPET? Dev Integration QA Stage Live
  7. 7. WHAT WE'RE AFTER Confidence that everything will work correctly in production Consistency between environments
  8. 8. OPS AND DEVS COOPERATING Previously: Devs built stuff Later, Ops came and built production infrastructure This caused many IT problems The solution?
  9. 9. OPSVELOPMENT
  10. 10. OPS AND DEVS WORKING TOGETHER Ops need to be involved in development planning process Puppet modules and manifests should be selected/built as part of the development process
  11. 11. DEVELOP ON PUPPET PROVISIONED ENVIRONMENTS As early as possible, all dev should be done on systems built from Puppet Puppet manifests get tested as part of the development process
  12. 12. VAGRANT Builds virtual machines, optionally from Puppet manifests Makes it easy to spin up short-lived dev instances Quick to get working Avoid ops being a blocker for dev
  13. 13. A WORKFLOW Development happens on Vagrant VM(s) Deployment to all shared environments happens via Jenkins
  14. 14. PUPPET CONFIG There should be only one set of Puppet manifests/modules Tested deployed and merged through software test environments
  15. 15. ONE SET OF MANIFESTS, MANY ENVIRONMENTS Different environments need different config Resource locations Settings
  16. 16. DEALING WITH DIFFERENT ENVIRONMENTS Hiera Removes the need for ugly if/else blocks Put anything that differers by environment in a separate file Can encrypt with hiera-gpg if data sensitive
  17. 17. HIERA.YAML :irrh: heacy -%evrnet {niomn} -cmo omn
  18. 18. VAGRANTFILE VgatcniueVGATIEAIVRIN d |ofg arn.ofgr(ARNFL_P_ESO) o cni| cni.mbx="ets4lc ofgv.o cno6-x" cni.mhsnm ="uptofeape ofgv.otae ppecn-xml" cni.mpoiin:uptd |upt ofgv.rvso ppe o ppe| ppe.aiet_ah uptmnfsspt ="uptmnfss ppe/aiet" ppe.aietfl uptmnfs_ie ="iep" st.p ppe.ouept uptmdl_ah ="uptmdls ppe/oue" ppe.ir_ofgpt ='uptheaym' uptheacni_ah ppe/ir.al ppe.pin = [-evrnet,"oadv] uptotos > "-niomn" lcle" ed n ed n
  19. 19. TO DEVELOP: Start of the day, dev runs v g a t u and gets the latest arn p environment Code/objects sit in a shared vagrant volume End of the day, or when new Puppet manifests/modules are available, v g a t d s r yis run arn eto
  20. 20. VAGRANT PROVISIONERS Avoid VirtualBox wherever possible Slow, prone to taking down host machine On Linux, vagrant-lxc is speedy VMWare Fusion for non-free fruit-based Unix
  21. 21. VAGRANT AND AWS Use Vagrant to bring up machines in AWS using v g a t a splugin arn-w Makes it easy to share work in progress Means VirtualBox doesn't crash your laptop Has cost implications
  22. 22. QA/STAGING ENVIRONMENTS IN AWS Merge to appropriate branch in git Jenkins takes over
  23. 23. ADVANTAGE OF AWS Great thing about AWS - we don't need to run our test environments all the time Have the environments only when you need them
  24. 24. TESTING VS LIVE Use the money saved to build better environments Minimise differences between testing and live In particular, test on environments with relevant HA as early as possible
  25. 25. SPEEDING UP THE PROCESS Some resources, in particular DBs can be slow to provision (30 mins plus) Could just run 24/7 One approach: pilot light provisioning
  26. 26. PILOT LIGHT PROVISIONING Tiers built using autoscaling groups Minimum instance count is 0 Jenkins sets desired capacity appropriately on deploy Reset to 0 via a recurring scheduled operation on ASG and/or Jenkins job
  27. 27. CONCLUSIONS Infrastructure development should run in parallel to software dev This means devs + ops must co-operate Minimise differences from production at all stages If a dev can't see the problem in their environment, you're much more likely to get woken up by it
  28. 28. QUESTIONS? COMMENTS? Sam Bashton sam@bashton.com Twitter: @bashtoni (Psst.. http://www.bashton.com/jobs/ )
  29. 29. REFERENCES, LINKS Vagrant vagrant-lxc hiera-gpg Masterless Puppet + AWS
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×