Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20180607 master your vms with vagrant


Published on

How to create, provision and manage VM's with Oracle installations under a different Linux user (oracle) than the Vagrant provision user (vagrant). With code examples.

Published in: Technology
  • Be the first to comment

20180607 master your vms with vagrant

  1. 1. DARWIN IT-PROFESSIONALS IT Driven Evolution Master your VM’s withVagrant Martien van den Akker 2018
  2. 2. Who I am 2© Darwin IT-Professionals B.V.
  3. 3. Per november ‘17: OracleACE 3© Darwin IT-Professionals B.V.
  4. 4. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning 1 2 3 4 6 Agenda 7 5 4© 2018 Darwin IT-Professionals B.V.
  5. 5. DarwinIT-Professionals AN INTRODUCTION TO VIRTUALIZATION 5© 2018 Darwin IT-Professionals B.V.
  6. 6. Quick Introductionto Virtualization • VMWare first to introduce virtualization on Intel platform • VirtualMachines – Behave like real computer servers – Highly movable because they’re ‘just a bunch of files’ – Not dependend on hardware changes – Allow for sharing hardware, better allocation and use of resources • Started with VMWare Workstation and later Server and Player around 2002, 2003. • Since 2009 Oracle owns VirtualBox (Sun Microsystems acquisition) 6© 2018 Darwin IT-Professionals B.V.
  7. 7. Quick Introductionto Virtualization • Never install server software right on the laptop – New laptop just means installing hypervisor and moving the VMs – In 2005/2006 investigated and implemented Oracle B2B: VMWare allows for multiple instances on same laptop. Or on different servers to mock different trading partners. • Only install JDeveloper and SQL developer on host (laptop) • VMWare, VirtualBox, etc. implement virtualization on HardWare level • Docker, etc. implement virtualization on OS Level 7© 2018 Darwin IT-Professionals B.V.
  8. 8. Quick Introductionto Virtualization • But: – Creation of new VM’s is a tedious job – New OS update or new release of software asks for new VM installations – Installations/configurations done by hand, hard to reproduce – Multiple versions of VMs with different states to be archived/catalogued – VMs can grow big – After period of not-use: state of VM is unclear – Every colleague creates a VM in his/her own way: lack of standardization – … 8© 2018 Darwin IT-Professionals B.V.
  9. 9. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning 1 2 3 4 6 Agenda 7 5 9© 2018 Darwin IT-Professionals B.V.
  10. 10. Vagrant 10© 2018 Darwin IT-Professionals B.V.
  11. 11. • Open-source software product for building and maintaining portable virtual software development environments • Simplifies software configuration management of virtualizations • To increase development productivity Vagrant: what? 11© 2018 Darwin IT-Professionals B.V.
  12. 12. Creating and maintaining VM’s is a tedious job: • Upgrades of OS • Upgrades of Software • States of the configuration Vagrant: • Automates creation of VM’s • Automates provisioning of VM’s • Abstracts – the configuration of the VM – the installation/setup of the software within the VM Vagrant: why? 12© 2018 Darwin IT-Professionals B.V.
  13. 13. Vagrant:How? 13© 2018 Darwin IT-Professionals B.V. Base Box (Repo/Url/File) Vagrantfile Shell Scripts/ Cookbooks/ Recipies Vagrant Provisioners Chef Puppet Ansible Shell VirtualBox VMWare Docker Providers Appliance
  14. 14. • Project folder – Contains Vagrantfile with all the configuration • BaseBox – Downloadable from repo, or custom box – Contains a VM, with OS installed in it – Vagrant user with sudo rights, and insecure key (replaced on startup) – SSH daemon running – NAT network adapter VagrantProjectIngredients 14© 2018 Darwin IT-Professionals B.V.
  15. 15. Building block concepts: • Providers: – VirtualBox – Docker – Vmware – AWS Services to set up and create VMs VagrantArchitecture Components 15© 2018 Darwin IT-Professionals B.V. • Provisioners: – Shell – Ansible – Puppet – Chef Tools to customize the configuration of VM
  16. 16. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning 1 2 3 4 6 Agenda 7 5 16© 2018 Darwin IT-Professionals B.V.
  17. 17. DarwinIT-Professionals COOK A VAGRANT BASE BOX IN YOUR KITCHENETTE 17© 2018 Darwin IT-Professionals B.V.
  18. 18. What is a basebox? • A base box is a Vagrant template to build a VM • Downloadable from Vagrant Cloud: (Just provide a name) • Downloadable from an URL • Local file • Added in local vagrant repo on first use 18© 2018 Darwin IT-Professionals B.V.
  19. 19. Base box requirements • An OS: I use Oracle Linux 7 Update 5 for this story • A vagrant user – public insecure key – add vagrant to the sudoers list • A started ssh daemon • NAT Adapter as the first one • VirtualBox GuestAdditions installed Password of root convention: vagrant 19© 2018 Darwin IT-Professionals B.V.
  20. 20. Base box buildoptions • Vagrant recommends ‘Packer’ – Automates creation of base boxes ( – Has its own requirements – For full E2E lifecycle – Regularly building base boxes on a daily base (on nightly builds for instance) • Vagrant has a built in package command – Incidentally creating base boxes – I’ll use this for the story 20© 2018 Darwin IT-Professionals B.V.
  21. 21. Base box buildsteps – 1 DownloadOS • I used the latest OL7U5 • Download Oracle Linux from 21© 2018 Darwin IT-Professionals B.V.
  22. 22. Base box buildsteps – 2 Create VM • VirtualBox with Ext pack and Vagrant assumed installed • Create the VM – 10240 MB memory and a 128GB dynamically allocated virtual disk, 4 CPUs – Add downloaded ISO – Start VM 22© 2018 Darwin IT-Professionals B.V.
  23. 23. Base box buildsteps – 2 InstallOS • KaSaP principle: Keep as Slim as Possible – Just select Server with GUI – Chang the network hostname to darlin-vce.darwin-it.local – Set root password to vagrant – Create vagrant user with password vagrant • Finish installation – Answer licensing question 23© 2018 Darwin IT-Professionals B.V.
  24. 24. Base box buildsteps – 2 InstallOS • Finish OS installation – Answer licensing question – Enable ‘Automatically connect to this network when it is available’ on the General tab 24© 2018 Darwin IT-Professionals B.V.
  25. 25. Base box buildsteps – 3 InstallGuest Additions • Install Kernel Packages (more fun to do afterwards) – Stop packagekit – Install the packages kernel-uek-devel kernel-uek-devel-4.1.12- 112.16.4.el7uek.x86_64 25© 2018 Darwin IT-Professionals B.V.
  26. 26. Base box buildsteps – 3 InstallGuest Additions • Insert Guest Additions 26© 2018 Darwin IT-Professionals B.V.
  27. 27. Base box buildsteps – 4 Configure Vagrant user • Add line to /etc/sudoers file: • As vagrant user create a .ssh folder in the vagrant home folder and create file authorized_keys in it: • Insert the in secure key from 27© 2018 Darwin IT-Professionals B.V.
  28. 28. Base box buildsteps – 5 package the box • Base box VM is ready: package it. • You might want to create an OVA of it, so you can remove the base vm • To package box, use vagrant package command: – -- base: Name of VirtualBox VM to package – -- output: file path to the box to package box to create 28© 2018 Darwin IT-Professionals B.V.
  29. 29. Manage your base boxes • At startup the box is added to your local Vagrant Repo • List boxes with box list: • Files can be found in .vagrant.d/boxes folder your user profile • Can be removed using box remove: 29© 2018 Darwin IT-Professionals B.V.
  30. 30. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning 1 2 3 4 6 Agenda 7 5 30© 2018 Darwin IT-Professionals B.V.
  31. 31. DarwinIT-Professionals BASEBOXREADY? LET’S CREATE A BOX! 31© 2018 Darwin IT-Professionals B.V.
  32. 32. Create a project structure • Create a project structure with – boxes – base boxes – projects – Vagrant projects – Stage – Stage folder installation-binaries, for database, Weblogic, Java, FusionMiddleware and so on. • Create a project folder for new VM • Open a command window (Vagrant assumed installed) • A clear project structure makes it easy to copy&paste and share projects and refer to components 32© 2018 Darwin IT-Professionals B.V.
  33. 33. Create a project structure • Initialize Project folder: • With option -m or --minimal setting a just enough vagrant file is created, without comments 33© 2018 Darwin IT-Professionals B.V.
  34. 34. EditVagrantfile 34© 2018 Darwin IT-Professionals B.V. Vagrantfile Configuration (version 2) • Define base box to use, multi-machine naming • SSH config Provider Configuration • Set VM properties (Memory, CPU, etc) • Toggle Bidirectional Clipboard/Drag&Drop • Define and attach extra storage/media Provisioning • Call shell script or Chef/Puppet/Ansible recipies/playbooks, etc. • Configuration version – 2 relates to version 2 of the Vagrant configuration options – Can have multiple configuration blocks – Ends with end • Place Global variables at the top of the file – Easy tweaking/configuration – Readable, meaningful settings – Adds to auto-commenting project file Globals (Recommended)
  35. 35. Introduce globals • Start with global variables for easy config: • BOX_URL: link to the base box created in former section • VMS_HOME: location where VM will be created. To have extra disk files in the same folder. 35© 2018 Darwin IT-Professionals B.V.
  36. 36. VM Configuration • Name the base-box using globals • Add the lines: – config.vm.box_url: directive refers to the base box used • Just give a name to the box in the Vagrant Cloud (see earlier) • Or link to the boxes folder – config.vm.define: allows to define multi-machine definitions in one Vagrant project, like Middleware and Database in separate VMs. 36© 2018 Darwin IT-Professionals B.V.
  37. 37. EditVagrantfile:SSHvagrant user • Vagrant uses SSH to connect to VM and provision it. • Can be configured using config.ssh.* properties: • Password should be optional, because of the injected insecure key. • At first startup insecure key will be replaced by generated secure key. • Config.ssh.port: port Vagrant uses as a local port forwarded to the internal ssh port. 37© 2018 Darwin IT-Professionals B.V.
  38. 38. Provider config • Provider configured using following block: 38© 2018 Darwin IT-Professionals B.V.
  39. 39. Provider config • Properties in the block: 39© 2018 Darwin IT-Professionals B.V. Property Meaning Name of the VM to create. This is how the VM will appear in VirtualBox. vb.gui Toggles the appearance of the UI of the VM vb.memory This sets the memory available to the VM. In the global variables I have set it to 12GB (12*1024 MB) vb.cpus Number of CPU cores available to the VM. In the globals set to 4.
  40. 40. Provider config – Create 2nd disk • vb.customize to an API to the VboxManage utility of VirtualBox: – modifyvm: modify vm properties like --clipboard and --draganddrop to bidirectional – createmedium: create a new disk: – --format vdi: Virtual Disk Image – --variant Standard: dynamically allocated file – unless File.exists – only create the file if it does not exist yet: 40© 2018 Darwin IT-Professionals B.V. vb.customize [ "createmedium", "disk", "--filename", VM_DISK2, "--format", "vdi", "--size", VM_DISK2_SIZE , "--variant", "Standard" ]
  41. 41. Provider config – attach2nd disk • vb.customize to an API to the VboxManage utility of VirtualBox: – storageattach: attach the disk to the VM – --storagectl SATA: storage controller to add the disk to – --port, --device: port and device numbers on the controller – --type hdd – we’re adding an harddisk here – --medium: the medium to add – :id: Special variable that refers to the VM that is created in VirtualBox 41© 2018 Darwin IT-Professionals B.V. vb.customize [ "storageattach", :id , "--storagectl", "SATA", "--port", "2", "--device", "0", "--type", "hdd", "--medium", VM_DISK2]
  42. 42. Shared/Synchedfolders • Vagrant by default creates a folder-link or shared folder, a so-called Synched Folder • Vagrant project folder (the one with Vagrantfile in it) mapped by default on /vagrant. • For stage folder we add: • Remember: 42© 2018 Darwin IT-Professionals B.V.
  43. 43. Provisioning • Vagrant allows for several provisioners like Puppet, Chef, Ansible, Salt. But a sample Shell snippet is provided in our Vagrantfile: • I replaced it with: 43© 2018 Darwin IT-Professionals B.V. • Only main steps, details in sub scripts • Script is run as vagrant • Can also use external scripts, even over URL
  44. 44. Up, up and up it goes…! • If everything went alright, you’re now ready to fire up your VM. • Then simply issue the following command in the command window: 44© 2018 Darwin IT-Professionals B.V.
  45. 45. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning 1 2 3 4 6 Agenda 7 5 45© 2018 Darwin IT-Professionals B.V.
  46. 46. DarwinIT-Professionals A BIT MOREADVANCEDPROVISIONING 46© 2018 Darwin IT-Professionals B.V.
  47. 47. Or actually, the question is: How do I provision using a provision user but not have it become the product owner? SO:… A bit more advancedprovisioning 47© 2018 Darwin IT-Professionals B.V. Compare Schema Owner user in database Compare SYSTEM or DBA-user in database
  48. 48. A quick introduction to Virtualization Vagrant Cook a Vagrant Base Box in your Kitchenette Base box ready? Let's create a box! A bit more advanced provisioning Provision but not owning 1 2 3 4 6 Agenda 7 5 48© 2018 Darwin IT-Professionals B.V.
  49. 49. The provisioning/notowningdilemma • The provisioning is done using the vagrant user. • The vagrant user is in the sudoers list -> can run a script using the permissions of another user; – sudo su - oracle -c "script“ does not do the trick -> vagrant still owns the files • It is still the running vagrant user who owns the resulting files and folders. • But oracle (or other install owner) is to own it. • Option: create a new base box with another install user? 49© 2018 Darwin IT-Professionals B.V.
  50. 50. Solution:run as another user • 50© 2018 Darwin IT-Professionals B.V.
  51. 51. Solution:run as another user • Must be run as root, but that's no problem since vagrant is in the sudoers list • Add following to the provisioning block: • Script left unexplained… • Database, Fusion Middleware, etc. or other tools requiring other users can be installed this way 51© 2018 Darwin IT-Professionals B.V.
  52. 52. DarwinIT-Professionals THANKYOU FORYOUR ATTENDANCE, PATIENCE AND ATTENTION 52© 2018 Darwin IT-Professionals B.V.
  53. 53. Q & A 53© Darwin IT-Professionals B.V.
  54. 54. DarwinIT-Professionals HELPFUL COMMANDS 54© 2018 Darwin IT-Professionals B.V.
  55. 55. Some other helpfulcommands • Basic vagrant commands: 55© 2018 Darwin IT-Professionals B.V. Command Meaning init Initializes current folder with an initial Vagrantfile package Packages a VM from provider into a box up Creates a vm from a box if non-existent, starts up the VM, and provisions it if not already done halt Stops the VM suspend Suspends the VM destroy Removes the VM
  56. 56. Some other helpfulcommands • Basic vagrant commands: 56© 2018 Darwin IT-Professionals B.V. Command Meaning ssh Starts a secure shell connection, logs on as vagrant user provision Reruns the provisioning of the VM box add This adds a box with the given address to Vagrant local repo box list This command lists boxes in Vagrant local repo box remove This command removes a box from Vagrant local repo box update Updates a box from an updated base box in local repo
  57. 57. DarwinIT-Professionals EXAMPLE PROVISION SCRIPTS 57© 2018 Darwin IT-Professionals B.V.
  58. 58. Prepare Oracle Linux • Install required packages • Install haveged to increase entropy for non-gui VMs • Add security limits for oracle user • Change kernel settings 58© 2018 Darwin IT-Professionals B.V.
  59. 59. Create filesystem • Create mountpoint • Create logical group and volume using System Storage Manager • Add mount point to fstab 59© 2018 Darwin IT-Professionals B.V.
  60. 60. Create Oracle user 60© 2018 Darwin IT-Professionals B.V. • Function to read oracle user properties from file • Create groups dba and oinstall • Create the user with the groups • Set the password from property file
  61. 61. Create Oracle user 61© 2018 Darwin IT-Professionals B.V. • Add oracle to sudoers • Create oraInventory • Create oraInst.loc