CloudOpen North America 2013: Vagrant & CFEngine

1,431 views

Published on

During this hands-on tutorial you will learn how to quickly provision local test/development/demo environments using Vagrant and Virtualbox. We will cover provisioning and configuring machines quickly using Vagrant and CFEngine. You will learn how Vagrant and Virtualbox can be used to bring up local development/test/demo environments. You will also learn how CFEngine can be leveraged to automate configuration of the environment after it has been initialized. You will take away a multi-vm test environment managed by CFEngine.

This tutorial targets technical people who need repeatable test environments and are comfortable using the Linux command-line. These environments can speed developer on-boarding, play a role in continuous integration, or just provide quick sandboxes for experimentation. No previous knowledge of Vagrant or CFEngine is required.

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,431
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

CloudOpen North America 2013: Vagrant & CFEngine

  1. 1. www.cfengine.com Vagrant & CFEngine CloudOpen North America 2013
  2. 2. Before we get started Is everyone in the right place? Has everyone installed Virtualbox 4.2.16 or later? VirtualBox --help | grep VirtualBox Has everyone installed Vagrant 1.2.7 or later? vagrant –version I have USB keys with Installers and content for the tutorial, if you don't have it yet please let me know.
  3. 3. Get to know each other Hi, I'm Nick SysAdmin > 10 Years Work at CFEngine Live in Lawrence, KS Twitter: @cmdln_ IRC: nickanderson Blog: http://www.cmdln.org Who are you? What do you do? Have you used Vagrant? Have you used CFEngine? Why did you choose this session and what do you hope to get from it?
  4. 4. 9/17/13 What is Vagrant? Tool to make working with development environments easy. Create, configure, destroy lightweight, reproducible, and portable environments. ● Created by Mitchell Hashimoto ● @mitchelh ● http://www.vagrantup.com
  5. 5. 9/17/13 How can it help? Developer on-boarding Quickly provision/decommission test environments in repeatable fashion Bug Validation Continuous Integration Ad-hoc Demos
  6. 6. 9/17/13 Provides common environment Designers Developers Operations QA
  7. 7. 9/17/13 Portable ● VirtualBox ● AWS ● VMware ● More ● https://github.com/mitchellh/vagrant/wiki/Available-Vagrant-Plugins
  8. 8. 9/17/13 So what's it made of?
  9. 9. 9/17/13 Vagrantfile
  10. 10. 9/17/13 Boxes ● Base operating system image ● Provider specific ● http://www.vagrantbox.es ● Use veewee or packer.io (build your own automatically) – Kickstart/preseed, postinstall scripts
  11. 11. 9/17/13 Magic ● Ssh port forwards ● Shared project folder /vagrant
  12. 12. 9/17/13 Useful Plugin ● vagrant-vbguest automatically installs the host's VirtualBox Guest Additions on the guest system. ● vagrant plugin install vagrant-vbguest ● https://github.com/dotless-de/vagrant -vbguest
  13. 13. 9/17/13 CFEngine Provisioner: Currently Undocumented ● am_policy_hub ● extra_agent_args – Extra arguments to pass to cf-agent executions ● classes – Additional classes to define when running cf-agent ● deb_repo_file – The apt repository configuration file to use for configuring the repository containing the CFEngine packages ● deb_repo_line – The line that specifys the repository to use for CFEngine packages ● files_path – Directory to copy on top of the default masterfiles ● force_bootstrap – If true, bootstrap the host even if it has been bootstrapped before ● install – Install CFEngine package from repository ● mode – “bootstrap” or “single_run”, determines whether CFEngine will be bootstrapped or just executed once on the host ● policy_server_address ● repo_gpg_key_url – http location of GPG key used for checking package signatures ● run_file – Standalone CFEngine policy file to upload and execute ● upload_path – Path to upload run_file ● yum_repo_file – The yum repository file to use when configuring the repository containing CFEngine packages ● yum_repo_url – The url of the repository containing the CFEngine packages ● package_name – The cfengine package name to install
  14. 14. 9/17/13 CFEngine Provisioner: Example Use
  15. 15. 9/17/13 Getting started ● vagrant box ● vagrant init ● vagrant status ● vagrant up ● vagrant ssh – vagrant ssh node ● vagrant destroy
  16. 16. 9/17/13 Daily Use vagrant up vagrant {destroy, halt, suspend} !-2
  17. 17. 9/17/13 This is fantastic!
  18. 18. 9/17/13 Build base boxes for all the things!
  19. 19. Black Hole
  20. 20. 9/17/13 Automating Vagrant Provisioning ● Ansible ● CFEngine ● Chef ● Puppet ● Salt Stack ● Shell Scripts ● MixnMatch!
  21. 21. 9/17/13 CFEngine ● IT infrastructure automation, compliance, and knowledge management framework ● Opensource and Commercial Software ● Originally written by Mark Burgess ● @markburgess_osl ● http://www.cfengine.com
  22. 22. 9/17/13 CFEngine History ● First released in 1993 ● CFEngine 2 released in 1998, self healing computer immunology. Added machine learning and anomaly detection. ● 2003 Promise Theory work began ● 2008 CFEngine 3 released. Integrates knowledge management and discovery mechanisms.
  23. 23. 9/17/13 CFEngine Properties ● Small
  24. 24. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search)
  25. 25. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable
  26. 26. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient
  27. 27. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient ● Declarative
  28. 28. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient ● Declarative
  29. 29. 9/17/13 Bootstrap a test environment
  30. 30. 9/17/13 Get Going ● Import vagrant basebox – cd resources/veewee – vagrant box add CFEngine_Training CFEngine_Training.box ● Bring up environment – vagrant status – vagrant up – vagrant status
  31. 31. 9/17/13 More Nodes! ● Increase nodes in Vagrantfile ● vagrant up ● vagrant ssh node00{1,2}
  32. 32. 9/17/13 CFEngine Design Center ● Community contributed reusable policy ● Curated Repository ● Cli and GUI (enterprise) clients
  33. 33. 9/17/13 Using cf-sketch to configure infrastructure ● Log in to your policy hub, locate the design center repository and access the cf-sketch shell – vagrant ssh hub – sudo -i – cd /vagrant/resources/design-center/tools/cf-sketch – ./cf-sketch.pl
  34. 34. 9/17/13 Configure Timezones ● search time ● info -v tzconfig ● install System::tzconfig ● define paramset System::tzconfig – Name: NO_Oslo_TZ – Timezone: Europe/Oslo – Zoneinfo: /usr/share/zoneinfo
  35. 35. 9/17/13 Configure Timezones ● search time ● install System::tzconfig ● define paramset System::tzconfig – Name: NO_Oslo_TZ – Timezone: Europe/Oslo – Zoneinfo: /usr/share/zoneinfo
  36. 36. 9/17/13 Configure Timezones Cont. ● define paramset System::tzconfig – Name: US_Central_TZ – Timezone: US/Central – Zoneinfo: /usr/share/zoneinfo
  37. 37. 9/17/13 Activate and Deploy Timezone Configuration ● activate System::tzconfig NO_Oslo_TZ hub ● activate System::tzconfig US_Central_TZ node001 ● deploy
  38. 38. 9/17/13 Editor War! ● Which side are you on? ● services/editor_war.cf
  39. 39. 9/17/13 Wage War ● Remove Disallowed Packages – vagrant ssh hub – watch rpm -q emacs-nox – Uncomment disallowed_packages to activate policy. Watch it get fixed. ● Install Required Packages – watch rpm -q vim-enhanced – Uncomment required_packages to activate policy
  40. 40. 9/17/13 The Books ● Learning CFEngine 3 – Diego Zamboni ● Vagrant Up and Running – Mitchell Hashimoto
  41. 41. 9/17/13 Questions/Discussion?
  42. 42. 9/17/13 Thank You!
  43. 43. www.cfengine.com Vagrant & CFEngine CloudOpen North America 2013
  44. 44. Before we get started Is everyone in the right place? Has everyone installed Virtualbox 4.2.16 or later? VirtualBox --help | grep VirtualBox Has everyone installed Vagrant 1.2.7 or later? vagrant –version I have USB keys with Installers and content for the tutorial, if you don't have it yet please let me know.
  45. 45. Get to know each other Hi, I'm Nick SysAdmin > 10 Years Work at CFEngine Live in Lawrence, KS Twitter: @cmdln_ IRC: nickanderson Blog: http://www.cmdln.org Who are you? What do you do? Have you used Vagrant? Have you used CFEngine? Why did you choose this session and what do you hope to get from it?
  46. 46. 9/17/13 What is Vagrant? Tool to make working with development environments easy. Create, configure, destroy lightweight, reproducible, and portable environments. ● Created by Mitchell Hashimoto ● @mitchelh ● http://www.vagrantup.com Questions? Stop me
  47. 47. 9/17/13 How can it help? Developer on-boarding Quickly provision/decommission test environments in repeatable fashion Bug Validation Continuous Integration Ad-hoc Demos
  48. 48. 9/17/13 Provides common environment Designers Developers Operations QA Really any person that needs to have a functional test environment.
  49. 49. 9/17/13 Portable ● VirtualBox ● AWS ● VMware ● More ● https://github.com/mitchellh/vagrant/wiki/Available-Vagrant-Plugins Take it with you (offline) VirtualBox Vmware Use someone else's infrastructure AWS Rackspace
  50. 50. 9/17/13 So what's it made of?
  51. 51. 9/17/13 Vagrantfile ●Vagrantfile describes machine configurations ●Syntax is Ruby, but knowledge of the Ruby language is not necessary. It's mostly simple variable assignment.
  52. 52. 9/17/13 Boxes ● Base operating system image ● Provider specific ● http://www.vagrantbox.es ● Use veewee or packer.io (build your own automatically) – Kickstart/preseed, postinstall scripts Veewee written by Patrick Debois Packer.io written by Mitchell Hashimoto Leverage your existing infrastructure. Use the same (or very similar) kickstart/preseed and postinstall scripts that you use in your production env.
  53. 53. 9/17/13 Magic ● Ssh port forwards ● Shared project folder /vagrant Automatically forwards Local port to 22 on the host Automatically mounts the directory that the Vagrantfile lives in (Vagrant Project Dir) on each host. Add your own custom Problems with automagic? Check the tools version. Not required to match but it helps
  54. 54. 9/17/13 Useful Plugin ● vagrant-vbguest automatically installs the host's VirtualBox Guest Additions on the guest system. ● vagrant plugin install vagrant-vbguest ● https://github.com/dotless-de/vagrant -vbguest ● Sometimes there are issues if the version of virtualbox tools does not match the currently running version. ● This plugin will detect if the guest tools are outdated, download build, install, restart the guest ● If you're lucky, vagrant-vbguest does not require any configurations.
  55. 55. 9/17/13 CFEngine Provisioner: Currently Undocumented ● am_policy_hub ● extra_agent_args – Extra arguments to pass to cf-agent executions ● classes – Additional classes to define when running cf-agent ● deb_repo_file – The apt repository configuration file to use for configuring the repository containing the CFEngine packages ● deb_repo_line – The line that specifys the repository to use for CFEngine packages ● files_path – Directory to copy on top of the default masterfiles ● force_bootstrap – If true, bootstrap the host even if it has been bootstrapped before ● install – Install CFEngine package from repository ● mode – “bootstrap” or “single_run”, determines whether CFEngine will be bootstrapped or just executed once on the host ● policy_server_address ● repo_gpg_key_url – http location of GPG key used for checking package signatures ● run_file – Standalone CFEngine policy file to upload and execute ● upload_path – Path to upload run_file ● yum_repo_file – The yum repository file to use when configuring the repository containing CFEngine packages ● yum_repo_url – The url of the repository containing the CFEngine packages ● package_name – The cfengine package name to install The CFEngine provisioner is currently undocumented. This is a great opportunity for someone to contribute. I already did part of the work right here in this slide ;) Options for Package Source/install Extra Arguments Bootstrap or Standalone oneshot policy
  56. 56. 9/17/13 CFEngine Provisioner: Example Use
  57. 57. 9/17/13 Getting started ● vagrant box ● vagrant init ● vagrant status ● vagrant up ● vagrant ssh – vagrant ssh node ● vagrant destroy
  58. 58. 9/17/13 Daily Use vagrant up vagrant {destroy, halt, suspend} !-2 There are more commands, but you use vagrant up to bring up an environment, and vagrant destroy to delete the vms.
  59. 59. 9/17/13 This is fantastic!
  60. 60. 9/17/13 Build base boxes for all the things! You can have too much of a good thing.
  61. 61. Black Hole There are so many places where configurations can hide. I think of Vms as kind of a black hole of knowledge.
  62. 62. 9/17/13 Automating Vagrant Provisioning ● Ansible ● CFEngine ● Chef ● Puppet ● Salt Stack ● Shell Scripts ● MixnMatch! Automate configuration on top of base image. There are valid reasons for baking config into a basebox, usually for speed of deployment. Not a replacement for good configuration management.
  63. 63. 9/17/13 CFEngine ● IT infrastructure automation, compliance, and knowledge management framework ● Opensource and Commercial Software ● Originally written by Mark Burgess ● @markburgess_osl ● http://www.cfengine.com
  64. 64. 9/17/13 CFEngine History ● First released in 1993 ● CFEngine 2 released in 1998, self healing computer immunology. Added machine learning and anomaly detection. ● 2003 Promise Theory work began ● 2008 CFEngine 3 released. Integrates knowledge management and discovery mechanisms. CFEngine has a solid history. Its been around for 20 years. Runs on over 10 million servers in over 10 thousand companies Promises are a declaration of intent
  65. 65. 9/17/13 CFEngine Properties ● Small CFEngine is written in C ~ 100k lines of code (remember its a 20 year old project) ~ 5M single package install ~ 15-25M memory consumption (depends on your policy of course)
  66. 66. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) Security is a core focus. The voluntary cooperation principal of Promise Theory and the pull model are important for this. Great track record CFEngine hasn't had a published security vulnerability since 2005 (CFEngine 2) 0 since CFEngine 3 was released in 2009
  67. 67. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable Because its written in C it runs on just about anything. Linux, BSDs, AIX, HPUX, Solaris even Windows Storage Devices (Qnap) Switches (Cisco, Arista, Juniper) Embedded Devices Raspburry Pi Robot bottom of the ocean Water testing devices in fields with cows Laser cutters (that make puppets)
  68. 68. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient CFEngine works when other things are broken. CFEngine tries to fix itself – failsafe.cf Decisions are made by the agents running on individual hosts. If the network is down they continue to apply the policy they have. These policies can be extremely dynamic since all decisions are made by the individual agent. They can use external sources of information if desired or required. Convergence – continual repair of system state toward desired specification If something can't be fixed track it and move on (usually) If installing httpd fails, it could continue on and ensure that SSH is hardened. Or if you desire all execution could stop at that point.
  69. 69. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient ● Declarative CFEngines policy language is declarative in nature. This allows you to focus on the goals of how things should be and converge towards this desired state. It works kind of like a GPS. It doesn't matter where you start, it will continually re-route to reach the destination. For example Apache promises to be installed on webservers. Not install apache on host x,y,z Httpd process promises to be running in production during non maintenance hours Sshd process promises to not be running, and completely fire-walled off when the number of SSH sessions into or out of a host are 3 standard deviations higher than normal
  70. 70. 9/17/13 CFEngine Properties ● Small ● Secure (http://web.nvd.nist.gov/view/vuln/search) ● Portable ● Resilient ● Declarative CFEngines policy language is declarative in nature. This allows you to focus on the goals of how things should be and converge towards this desired state. It works kind of like a GPS. It doesn't matter where you start, it will continually re-route to reach the destination. For example Httpd config file promises to have this configuration for hosts running application x Httpd process promises to be running on web servers Sshd process promises to not be running when the number of SSH sessions into or out of a host are 3 standard deviations higher than normal
  71. 71. 9/17/13 Bootstrap a test environment Examine Vagrantfile Shell provisioner to prep the environment for offline use. Dynamic multi-vm configuration Host only network for vms to communicaate on Forward ports Synced files for hub masterfiles (normally, you would update your masterfiles from a version control repository) CFEngine policy Splay set to 0 Runs every minute (body executor control) Emails root@localhost Pre-written demo policy
  72. 72. 9/17/13 Get Going ● Import vagrant basebox – cd resources/veewee – vagrant box add CFEngine_Training CFEngine_Training.box ● Bring up environment – vagrant status – vagrant up – vagrant status We need to add it manually because conference INTERNET
  73. 73. 9/17/13 More Nodes! ● Increase nodes in Vagrantfile ● vagrant up ● vagrant ssh node00{1,2} Increase nodes to 1 or 2 (dependent on resources) Verify that you can ssh to them Check out shared directory support Look in /vagrant Update a file from inside the vm, check from workstation, vice versa
  74. 74. 9/17/13 CFEngine Design Center ● Community contributed reusable policy ● Curated Repository ● Cli and GUI (enterprise) clients When I am talking about paths to cfengine configuration files in these examples they are relative to masterfiles So the synced vagrant directory is resources/synced_masterfiles Edit there and the policy will get synchronized to the hubs masterfiles directory
  75. 75. 9/17/13 Using cf-sketch to configure infrastructure ● Log in to your policy hub, locate the design center repository and access the cf-sketch shell – vagrant ssh hub – sudo -i – cd /vagrant/resources/design-center/tools/cf-sketch – ./cf-sketch.pl When I am talking about paths to cfengine configuration files in these examples they are relative to masterfiles So the synced vagrant directory is resources/overlay_var_cfengine/masterfiles Edit there and the policy will get synchronized to the hubs masterfiles directory
  76. 76. 9/17/13 Configure Timezones ● search time ● info -v tzconfig ● install System::tzconfig ● define paramset System::tzconfig – Name: NO_Oslo_TZ – Timezone: Europe/Oslo – Zoneinfo: /usr/share/zoneinfo You may want to have terminal open
  77. 77. 9/17/13 Configure Timezones ● search time ● install System::tzconfig ● define paramset System::tzconfig – Name: NO_Oslo_TZ – Timezone: Europe/Oslo – Zoneinfo: /usr/share/zoneinfo When I am talking about paths to cfengine configuration files in these examples they are relative to masterfiles So the synced vagrant directory is resources/overlay_var_cfengine/masterfiles Edit there and the policy will get synchronized to the hubs masterfiles directory
  78. 78. 9/17/13 Configure Timezones Cont. ● define paramset System::tzconfig – Name: US_Central_TZ – Timezone: US/Central – Zoneinfo: /usr/share/zoneinfo When I am talking about paths to cfengine configuration files in these examples they are relative to masterfiles So the synced vagrant directory is resources/overlay_var_cfengine/masterfiles Edit there and the policy will get synchronized to the hubs masterfiles directory
  79. 79. 9/17/13 Activate and Deploy Timezone Configuration ● activate System::tzconfig NO_Oslo_TZ hub ● activate System::tzconfig US_Central_TZ node001 ● deploy When I am talking about paths to cfengine configuration files in these examples they are relative to masterfiles So the synced vagrant directory is resources/overlay_var_cfengine/masterfiles Edit there and the policy will get synchronized to the hubs masterfiles directory
  80. 80. 9/17/13 Editor War! ● Which side are you on? ● services/editor_war.cf Stop here, take a look at the file. Who can tell what the policy is doing without having it explained to them first?
  81. 81. 9/17/13 Wage War ● Remove Disallowed Packages – vagrant ssh hub – watch rpm -q emacs-nox – Uncomment disallowed_packages to activate policy. Watch it get fixed. ● Install Required Packages – watch rpm -q vim-enhanced – Uncomment required_packages to activate policy How was this policy executed? See body common control inputs and bundlesequence. Try playing around and manually removing packages
  82. 82. 9/17/13 The Books ● Learning CFEngine 3 – Diego Zamboni ● Vagrant Up and Running – Mitchell Hashimoto
  83. 83. 9/17/13 Questions/Discussion?
  84. 84. 9/17/13 Thank You!

×