Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Puppet and AWS is easy …… ?
Cara Crawford and Scott Coulton
Who are we?
DevOps/Infrastructure Manager
2	
  
Cara Crawford
•  8 + years development experience
•  6 + years operational...
Who are we?
DevOps Solution Architect
3	
  
Scott Coulton
•  10 + years hosting solutions experience
•  Extensive experien...
Who are we?
4	
  
Healthdirect Australia manages the following healthcare
service:
•  Healthdirect nurse helpline and heal...
What we were trying to solve
August 2013
5	
  
•  Majority of servers located in AWS
•  Manual deployments
•  Cloning for ...
How did we solve it
6	
  
First Iteration – Nov 2013
•  Basic Combination approach of Boto and Python for AWS scripting
• ...
Our Environment
7	
  
5 development environments

3 Puppet masters
Dynamically controlling the 
VPC environments they are ...
So it all looks easy, Right !!!!!
8	
  
What have we learnt ?
9	
  
A solid foundation
-  Development … Where to start ???
-  Unit and acceptance testing framewor...
Development, Where to start ?
10	
  
If you have never written a module start here !!!
11	
  
h#ps://forge.puppetlabs.com/	
  
Start with a solid development foundation !!!
12	
  
h#ps://github.com/mitchellh/vagrant	
  
A	
  must	
  have	
  for	
  a...
If you have not developed Vagrant framework , Here are is a good one
13	
  
h#ps://github.com/adrienthebo/oscar	
  
What we have done with Vagrant
14	
  
•  Re	
  useable	
  VagrantFile	
  with	
  variables	
  
•  Yaml	
  box	
  configuraM...
VagrantFile
15	
  
Yaml backend
16	
  
Unit testing is a must !!!
17	
  
•  Picks	
  up	
  human	
  error	
  
•  Tests	
  module	
  dependancies	
  
•  EssenMal	...
What do tests look like ?
18	
  
A	
  simple	
  example	
  
A	
  complex	
  example	
  
execute	
  the	
  test	
  
•  rake...
But wait what don't test cover ?
19	
  
•  Package	
  versions,	
  If	
  a	
  file	
  exists	
  etc	
  
So we need acceptan...
So now we have our working modules …
How do we integrate them with CD ?
20	
  
•  r10k	
  h#ps://github.com/puppetlabs/r10...
Puppet code deployment flow
21	
  
So some CD tricks and tips we use
22	
  
Every	
  module	
  has	
  its	
  own	
  repo	
  
So some CD tricks and tips we use
23	
  
Each	
  pipeline	
  runs	
  different	
  test	
  before	
  it	
  can	
  move	
  to...
AWS Deployment Tools
24	
  
…	
  And	
  because	
  we	
  are	
  not	
  big	
  fans	
  of	
  JSON	
  
h#ps://github.com/baz...
Example of the AWS Puppet module
25	
  
Benefits of the AWS Puppet module
26	
  
•  Anyone	
  on	
  the	
  team	
  can	
  develop	
  a	
  new	
  AWS	
  instance,	...
27	
  
What we are working on to take Puppet and
AWS to the next level
28	
  
29	
  
require => Class['docker']
docker-compose.yml.erb
What we are working on now to take Puppet to the next level
30	
  
Docker	
  scheduling	
  with	
  Puppet	
  
Dynamic	
  H...
So that was easy right ????
31	
  
Questions
http://www.healthdirect.gov.au
Upcoming SlideShare
Loading in …5
×

Puppet and AWS is Easy...?

1,008 views

Published on

Healthdirect presentation from Sydeny Puppet Camp

Published in: Technology

Puppet and AWS is Easy...?

  1. 1. Puppet and AWS is easy …… ? Cara Crawford and Scott Coulton
  2. 2. Who are we? DevOps/Infrastructure Manager 2   Cara Crawford •  8 + years development experience •  6 + years operational sysadmin experience •  Extensive experience across website solutions in both datacentre and cloud solutions across Linux and Windows environments     @cara_p
  3. 3. Who are we? DevOps Solution Architect 3   Scott Coulton •  10 + years hosting solutions experience •  Extensive experience in architecture across AWS, Puppet, Linux, Docker and all things open source Sc         h#ps://github.com/sco#y-­‐c   h#ps:// registry.hub.docker.com /repos/sco#yc/   h#ps://forge.puppetlabs.com/ sco#yc   @sco#coulton  
  4. 4. Who are we? 4   Healthdirect Australia manages the following healthcare service: •  Healthdirect nurse helpline and health information •  After hours GP helpline •  Pregnancy, Birth and Baby •  Mindhealthconnect •  National Health Service Directory •  My Aged Care •  Video Call Consulting    
  5. 5. What we were trying to solve August 2013 5   •  Majority of servers located in AWS •  Manual deployments •  Cloning for provisioning •  Inconsistent environments •  No CD •  No CI •  No Infrastructure as code •  No Automated testing •  Very limited automation     Deployments would take weeks or months with a high level of developer input onto cloned servers
  6. 6. How did we solve it 6   First Iteration – Nov 2013 •  Basic Combination approach of Boto and Python for AWS scripting •  Basic Opensource Puppet for configuration, server management Third Iteration – Jan 2015 •  AWS Puppet module with cloud formation •  Advanced puppet coding standards with testing •  Container services and schedulers Second Iteration – Feb 2014 •  Improvements to scripts •  Additional Puppet coding standards including upgrading to Enterprise •  Heading towards 100% Puppet coverage
  7. 7. Our Environment 7   5 development environments 3 Puppet masters Dynamically controlling the VPC environments they are responsible for Each environment the access level reducing towards Stage and Prod being automated environments
  8. 8. So it all looks easy, Right !!!!! 8  
  9. 9. What have we learnt ? 9   A solid foundation -  Development … Where to start ??? -  Unit and acceptance testing frameworks -  CD with Puppet Deploying to AWS with Puppet -  Tips and tricks What we are working on now to take Puppet to the next level
  10. 10. Development, Where to start ? 10  
  11. 11. If you have never written a module start here !!! 11   h#ps://forge.puppetlabs.com/  
  12. 12. Start with a solid development foundation !!! 12   h#ps://github.com/mitchellh/vagrant   A  must  have  for  any  Puppet  development     •  Quick  deployments   •  A  massive  community     •  Supports  any  OS  you  do     •  You  can  deploy  a  single  machine  or  100     •  It  is  as  complex  or  simple  as  you  need  it  to  be     •  Its  all  Ruby      
  13. 13. If you have not developed Vagrant framework , Here are is a good one 13   h#ps://github.com/adrienthebo/oscar  
  14. 14. What we have done with Vagrant 14   •  Re  useable  VagrantFile  with  variables   •  Yaml  box  configuraMon     •  MulM-­‐node  test  environments  with  both  open  source  and  Puppet  Enterprise     •  Full  Hiera  integraMon       •  We  have  a  full  replicaMon  of  producMon  on  our  laptops      
  15. 15. VagrantFile 15  
  16. 16. Yaml backend 16  
  17. 17. Unit testing is a must !!! 17   •  Picks  up  human  error   •  Tests  module  dependancies   •  EssenMal  for  the  CD/CI  process       Tools for the job h#p://rspec-­‐puppet.com/   h#ps://github.com/rodjek/rspec-­‐puppet  
  18. 18. What do tests look like ? 18   A  simple  example   A  complex  example   execute  the  test   •  rake  spec_prep  &&  rspec  
  19. 19. But wait what don't test cover ? 19   •  Package  versions,  If  a  file  exists  etc   So we need acceptance testing •  This  where  Vagrant  comes  back  in     Run the module !!!! Puppetlabs  have  a  very  good  framework  for  this  h#ps://github.com/puppetlabs/beaker  
  20. 20. So now we have our working modules … How do we integrate them with CD ? 20   •  r10k  h#ps://github.com/puppetlabs/r10k  is  a  good   place  to  start.   This  will  give  you  a  great  transport  mechanism  from  source   control  to  the  Puppet  master.   But what if we are using a CD tool (ie Jenkins etc) ?
  21. 21. Puppet code deployment flow 21  
  22. 22. So some CD tricks and tips we use 22   Every  module  has  its  own  repo  
  23. 23. So some CD tricks and tips we use 23   Each  pipeline  runs  different  test  before  it  can  move  to  the  next  
  24. 24. AWS Deployment Tools 24   …  And  because  we  are  not  big  fans  of  JSON   h#ps://github.com/bazaarvoice/cloudformaMon-­‐ruby-­‐dsl  
  25. 25. Example of the AWS Puppet module 25  
  26. 26. Benefits of the AWS Puppet module 26   •  Anyone  on  the  team  can  develop  a  new  AWS  instance,  elb  etc   •  Out  of  the  box  we  have  great  unit  tesMng  coverage   •  On  every  Puppet  run  reverts  the  environment  to  the  desired  state   Number  1  2p  for  AWS   h#ps://docs.puppetlabs.com/guides/rest_api.html   The  HTTP  API   AWS  SQS  +  Puppetlabs  HTTP  API  =  Happy  engineers      
  27. 27. 27   What we are working on to take Puppet and AWS to the next level
  28. 28. 28  
  29. 29. 29   require => Class['docker'] docker-compose.yml.erb
  30. 30. What we are working on now to take Puppet to the next level 30   Docker  scheduling  with  Puppet   Dynamic  Hiera  backend  and  service  discovery  
  31. 31. So that was easy right ???? 31  
  32. 32. Questions http://www.healthdirect.gov.au

×