Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Let's Code our Infrastructure!

1,687 views

Published on

Creating and maintaining different environments can be difficult and a real time-sink. We'll see in this talk how you can automate this tasks by building and updating your development and production environments on demand using Chef, Vagrant, Docker and Amazon Web Services.

Published in: Technology
  • Be the first to comment

Let's Code our Infrastructure!

  1. 1. Let’s Code our Infrastructure
  2. 2. Who I am ● Oswald De Riemaecker
 ● Continuous S.A. CEO & Co-Founder o Agile Delivery specialist o Editor of the continuousphp PaaS
 ● « Agilist », Cloud Architect
  3. 3. Agile Culture ● Improvement in the development team o Better collaboration o Continuous improvement o All skills / No silos o Testing Framework and continuous integration
 ● Improvement in Agile product management o Iterative development o Features driven o Maximizing Value o MVP/Lean Startup
  4. 4. DevOps Culture ● Agile + Operation
 ● 3 main pillars: o Infrastructure as Code (IaC) o Continuous Delivery / Deployment o Culture of cooperation
  5. 5. Infrastructure as Code ● Coding machines resources of the infrastructure
 ● Coding and versioning the infrastructure
 ● Testing both resources and infrastructure
 ● On demand rebuilding machines and infrastructure
  6. 6. Organization My Infrastructure Your Infrastructure Their Infrastructure
  7. 7. Local Environments Development Testing Staging Production
  8. 8. Roles Load Balancer Application Servers Indexes Servers Databases Servers
  9. 9. Resources ● A resource is a part of the system and its configuration
 Apache MySQL Solr
  10. 10. Complexity ● Maintaining infrastructure is complex ● It lives and evolves, so you have to 
 o Add new resources o Update resources o Apply Security Patches o Change/Update OS o ...
  11. 11. Tools ● The answer to this:
  12. 12. Chef ● Chef is an open-source tool ● Management of automation systems and infrastructure ● Chef is based on Ruby
  13. 13. Chef ● Nodes represent the servers within your infrastructure
 ● Nodes can be: o physical servers o virtual servers o instances in the cloud
 ● Each node belongs to: o an organization o an environment o has one or more role(s) Node NodeNodeNode Node Node
  14. 14. Chef Client / Server Node NodeNodeNode Node Node Chef Server chef-client Resources
  15. 15. Chef Cookbooks ● The cookbooks contain artifacts of our resources Recipes Templates Files Attributes State of the resources
  16. 16. Chef Cookbooks ● The cookbooks contain artifacts of our resources Recipes Templates Files Attributes State of the resources Installation / configuration of resources
  17. 17. Chef Cookbooks ● The cookbooks contain artifacts of our resources Recipes Templates Files Attributes State of the resources Installation / configuration of resources Configuration templates
  18. 18. Chef Cookbooks ● The cookbooks contain artifacts of our resources Recipes Templates Files Attributes State of the resources Installation / configuration of resources Configuration templates Configuration/script files
  19. 19. Provisioning
  20. 20. Role Provisioning 
 Node Role: Database Chef Server Hey, I am role “Database” and my environment is “Development” Chef clientRecipes Templates Attributes Files Cookbook Recipes Templates Attributes Files Cookbook
  21. 21. Infrastructure Example ● Web Server - Apache 2.4/PHP-5.5 ● Database Server - MySQL ● Index Server - Solr Web Server Solr MySQL
  22. 22. The Chef Community
  23. 23. Apache Cookbook Analysis ● Support many platforms ● Support different Apache versions ● Attributes: o dir, docroot_dir o log_dir, error_log, access_log o default_modules o version o ...
  24. 24. Cookbook support and dependencies Recipes Templates Attributs Files Metadata Dependancies
  25. 25. The attributes Recipes Templates Attributes Files Metadata
  26. 26. The recipes Recipes Templates Attributes Files Metadata cd recipes ls -l default.rb iptables.rb logrotate.rb mpm_prefork.rb mpm_worker.rb mod_php5.rb mod_*.rb
  27. 27. The Templates Recipes Templates Attributes Files Metadata
  28. 28. The Files Recipes Templates Attributes Files Metadata cd files/default/ ls -l apache2_module_conf_generate.pl
  29. 29. Let’s code our cookbooks
  30. 30. Role: Web Server / Env: Production Apache / php-5.5 Recipes Templates Attributes Files Metadata Apache2 Recipes Templates Attributes Files Metadata iptables Recipes Templates Attributes Files Metadata logrotate PRODUCTION: Replacement Attributes Node Role: WebServer R O L E
  31. 31. Chef Development Kit ● Installation of chef-dk
 o Berkself o Knife o ChefSpec o Test-Kitchen o Serverspec o Foodcritic
  32. 32. ● knife cookbook create mon_org-webserver Creating our web server cookbook
  33. 33. Adding our dependencies Dépendance(s)
  34. 34. ● berks init Initialization with Berkshelf
  35. 35. ● berks install Downloading dependencies
  36. 36. ● vi attributes/defaults.rb The Attributes
  37. 37. The Recipes
  38. 38. Testing our cookbook ● kitchen list
  39. 39. Testing our cookbook ● kitchen verify default-ubuntu-1404
  40. 40. ● knife cookbook upload mon_org-webserver Versioning and uploading our cookbook
  41. 41. Repeating the exercise for database and indexes roles
  42. 42. Provisioning our Sandbox Development
  43. 43. Role: Sandbox / Environment: Dev mon_org-webserver mon_org-index mon_org-database Composer NodeJs Recipes Templates Attributs Files Metadata Recipes Templates Attributs Files Metadata Recipes Templates Attributs Files Metadata Recipes Templates Attributs Files Metadata Recipes Templates Attributs Files Metadata mon_org-webserver mon_org-Index mon_org-database composer nodejs DEVELOPMENT: Replacement of Attributs Node Role: Sandbox R O L E
  44. 44. Creating our sandbox cookbook ● knife cookbook create mon_org-sanbox
  45. 45. Adding dependencies Dependancies
  46. 46. Downloading dependencies
  47. 47. The Recipes ● vi recipes/default.rb
  48. 48. ● vi attributes/default.rb The attributes Attributs
  49. 49. Testing our cookbook
  50. 50. ● Cookbooks by role of our infrastructure ● Specific Cookbook for our development environment o uses the cookbooks of our infrastructure roles
 ● We can reliably reproduce our infrastructure roles, it is:
 o Coded o Versioned o Tested o Deployed Provisioning Summary
  51. 51. Let’s create our environments
  52. 52. Local Creating our development environment Development
  53. 53. Vagrant ● Create and set up development environments
 ● Providers (Drivers) o Virtualbox/VMWare o Docker o Instance EC2
 ● Provisioning with: o Chef Client/Solo o Docker o Puppet Agent o …
  54. 54. Vagrantfile
  55. 55. Vagrant up
  56. 56. Immutable Infrastructure
  57. 57. Immutable Infrastructure ● Benefits: o Saving time when supplying o No more maintenance of the servers => Replacement o Static machine Image pre-built and tested o Easy rollback to a previous version V1 V2 V3 Data
  58. 58. Packer ● Packer
 o Builders (Amazon AMI, Microsoft Azure, DigitalOcean, Docker, Google Compute Engine, …) o Provisioners (Shell script, Chef client, Ansible, Puppet, Salt, …) o Post-processor (Docker-push, Vagrant, Atlas, …)
  59. 59. Packer Config Builders
  60. 60. Packer Config Provisioners
  61. 61. ● packer build packer.json Creating our image
  62. 62. Creating our image
  63. 63. Repeating the exercise for the index role
  64. 64. Creating our Production/Staging environment
  65. 65. AWS CloudFormation ● Easy way to create and manage an AWS resource
 ● CloudFormation takes care of dependencies
 ● Once the resources are deployed, you can edit and update in a predictable and controlled manner
 ● AWS CloudFormation supports the following resources:
 o Auto Scaling/Elastic Load Balancing o VPC o EC2 o RDS o IAM o …
  66. 66. VPC Private Subnet Public Subnet JSON Autoscale Web Web Web Autoscale Index Index Index
  67. 67. Cloud Formation Structure
  68. 68. Cloud Formation Structure
  69. 69. Cloud Formation Structure
  70. 70. Cloud Formation Structure
  71. 71. Cloud Formation Structure
  72. 72. Cloud Formation Structure
  73. 73. Creating our web server stack
  74. 74. Creating our web server stack
  75. 75. Creating our web server stack
  76. 76. Creating our web server stack
  77. 77. ● Machines Provisioning ● Immutable Infrastructure o Static Machine Images ● Using Machine Images in AWS Cloud Formation
 ● Cloud Formation environment parameters
 ● We can now use our artifacts to build our different environments Infrastructure Summary
  78. 78. Orchestrating our infrastructure Testing Development Vagrant Feature branch develop branch revision branch Staging Production CI/CD prepare test package prepare test package prepare test package
  79. 79. Questions?
  80. 80. Thank you! oswald_odr oswald@continuousphp.com

×