20180607 master your vms with vagrant

DARWIN IT-PROFESSIONALS
IT Driven Evolution
Master your VM’s withVagrant
Martien van den Akker
2018

Who I am
2© Darwin IT-Professionals B.V.

Per november ‘17: OracleACE

A quick introduction to Virtualization
Vagrant
Cook a Vagrant Base Box in your Kitchenette
Base box ready? Let's create a box!
A bit more advanced provisioning
1
2
3
4
6
Agenda
7
5
4© 2018 Darwin IT-Professionals B.V.

DarwinIT-Professionals
AN INTRODUCTION TO VIRTUALIZATION

Quick Introductionto Virtualization
• VMWare first to introduce virtualization on Intel platform
• VirtualMachines
– Behave like real computer servers
– Highly movable because they’re ‘just a bunch of files’
– Not dependend on hardware changes
– Allow for sharing hardware, better allocation and use of resources
• Started with VMWare Workstation and later Server and Player
around 2002, 2003.
• Since 2009 Oracle owns VirtualBox (Sun Microsystems acquisition)

• Never install server software right on the laptop
– New laptop just means installing hypervisor and moving the VMs
– In 2005/2006 investigated and implemented Oracle B2B: VMWare allows for
multiple instances on same laptop. Or on different servers to mock different
trading partners.
• Only install JDeveloper and SQL developer on host (laptop)
• VMWare, VirtualBox, etc. implement virtualization on HardWare
level
• Docker, etc. implement virtualization on OS Level

• But:
– Creation of new VM’s is a tedious job
– New OS update or new release of software asks for new VM installations
– Installations/configurations done by hand, hard to reproduce
– Multiple versions of VMs with different states to be archived/catalogued
– VMs can grow big
– After period of not-use: state of VM is unclear
– Every colleague creates a VM in his/her own way: lack of standardization
– …

Vagrant
1
2
3
4
6
Agenda
7
5

Vagrant

• Open-source software product for building and
maintaining portable virtual software development
environments
• Simplifies software configuration management of
virtualizations
• To increase development productivity
Vagrant: what?

Creating and maintaining VM’s is a tedious job:
• Upgrades of OS
• Upgrades of Software
• States of the configuration
Vagrant:
• Automates creation of VM’s
• Automates provisioning of VM’s
• Abstracts
– the configuration of the VM
– the installation/setup of the software within the VM
Vagrant: why?

Vagrant:How?
Base Box
(Repo/Url/File)
Vagrantfile
Shell Scripts/
Cookbooks/
Recipies
Vagrant
Provisioners
Chef Puppet Ansible Shell
VirtualBox
VMWare
Docker
Providers
Appliance

• Project folder
– Contains Vagrantfile with all the configuration
• BaseBox
– Downloadable from repo, or custom box
– Contains a VM, with OS installed in it
– Vagrant user with sudo rights, and insecure key (replaced on
startup)
– SSH daemon running
– NAT network adapter
VagrantProjectIngredients

Building block concepts:
• Providers:
– VirtualBox
– Docker
– Vmware
– AWS
Services to set up and
create VMs
VagrantArchitecture Components
• Provisioners:
– Shell
– Ansible
– Puppet
– Chef
Tools to customize the
configuration of VM

Vagrant
1
2
3
4
6
Agenda
7
5

COOK A VAGRANT BASE BOX IN YOUR
KITCHENETTE

What is a basebox?
• A base box is a Vagrant template to build a VM
• Downloadable from Vagrant Cloud:
https://app.vagrantup.com/boxes/search
(Just provide a name)
• Downloadable from an URL
• Local file
• Added in local vagrant repo on first use

Base box requirements
• An OS: I use Oracle Linux 7 Update 5 for this story
• A vagrant user
– public insecure key
– add vagrant to the sudoers list
• A started ssh daemon
• NAT Adapter as the first one
• VirtualBox GuestAdditions installed
Password of root convention: vagrant

Base box buildoptions
• Vagrant recommends ‘Packer’
– Automates creation of base boxes (http://www.packer.io)
– Has its own requirements
– For full E2E lifecycle
– Regularly building base boxes on a daily base (on nightly builds
for instance)
• Vagrant has a built in package command
– Incidentally creating base boxes
– I’ll use this for the story

Base box buildsteps – 1 DownloadOS
• I used the latest OL7U5
• Download Oracle Linux from edelivery.oracle.com

Base box buildsteps – 2 Create VM
• VirtualBox with Ext pack and Vagrant
assumed installed
• Create the VM
– 10240 MB memory and a 128GB dynamically
allocated virtual disk, 4 CPUs
– Add downloaded ISO
– Start VM

Base box buildsteps – 2 InstallOS
• KaSaP principle: Keep as Slim as Possible
– Just select Server with GUI
– Chang the network hostname to
darlin-vce.darwin-it.local
– Set root password to vagrant
– Create vagrant user with password vagrant
• Finish installation
– Answer licensing question

Base box buildsteps – 2 InstallOS
• Finish OS installation
– Answer licensing question
– Enable ‘Automatically connect to this
network when it is available’ on the
General tab

Base box buildsteps – 3 InstallGuest Additions
• Install Kernel Packages (more fun to do afterwards)
– Stop packagekit
– Install the packages kernel-uek-devel kernel-uek-devel-4.1.12-
112.16.4.el7uek.x86_64

Base box buildsteps – 3 InstallGuest Additions
• Insert Guest Additions

Base box buildsteps – 4 Configure Vagrant user
• Add line to /etc/sudoers file:
• As vagrant user create a .ssh folder in the vagrant home folder and
create file authorized_keys in it:
• Insert the in secure key from
https://github.com/hashicorp/vagrant/tree/master/keys:

Base box buildsteps – 5 package the box
• Base box VM is ready: package it.
• You might want to create an OVA of it, so you can remove the base
vm
• To package box, use vagrant package command:
– -- base: Name of VirtualBox VM to package
– -- output: file path to the box to package box to create

Manage your base boxes
• At startup the box is added to your local Vagrant Repo
• List boxes with box list:
• Files can be found in .vagrant.d/boxes
folder your user profile
• Can be removed using box remove:

Vagrant
1
2
3
4
6
Agenda
7
5

BASEBOXREADY? LET’S CREATE A BOX!

Create a project structure
• Create a project structure with
– boxes – base boxes
– projects – Vagrant projects
– Stage – Stage folder installation-binaries, for database, Weblogic, Java,
FusionMiddleware and so on.
• Create a project folder for new VM
• Open a command window
(Vagrant assumed installed)
• A clear project structure makes it easy to
copy&paste and share projects and refer to components

Create a project structure
• Initialize Project folder:
• With option -m or --minimal setting a just enough vagrant file is
created, without comments

EditVagrantfile
Vagrantfile
Configuration (version 2)
• Define base box to use, multi-machine naming
• SSH config
Provider Configuration
• Set VM properties (Memory, CPU, etc)
• Toggle Bidirectional Clipboard/Drag&Drop
• Define and attach extra storage/media
Provisioning
• Call shell script or Chef/Puppet/Ansible
recipies/playbooks, etc.
• Configuration version
– 2 relates to version 2 of the Vagrant
configuration options
– Can have multiple configuration blocks
– Ends with end
• Place Global variables at the top of the
file
– Easy tweaking/configuration
– Readable, meaningful settings
– Adds to auto-commenting project file
Globals (Recommended)

Introduce globals
• Start with global variables for easy
config:
• BOX_URL: link to the base box
created in former section
• VMS_HOME: location where VM
will be created. To have extra disk
files in the same folder.

VM Configuration
• Name the base-box using globals
• Add the lines:
– config.vm.box_url: directive refers to the base box used
• Just give a name to the box in the Vagrant Cloud (see earlier)
• Or link to the boxes folder
– config.vm.define: allows to define multi-machine definitions in one Vagrant
project, like Middleware and Database in separate VMs.

EditVagrantfile:SSHvagrant user
• Vagrant uses SSH to connect to VM and provision it.
• Can be configured using config.ssh.* properties:
• Password should be optional, because of the injected insecure key.
• At first startup insecure key will be replaced by generated secure
key.
• Config.ssh.port: port Vagrant uses as a local port forwarded to the
internal ssh port.

Provider config
• Provider configured using following block:

Provider config
• Properties in the block:
Property Meaning
vb.name Name of the VM to create. This is how the VM will appear in VirtualBox.
vb.gui Toggles the appearance of the UI of the VM
vb.memory This sets the memory available to the VM. In the global variables I have
set it to 12GB (12*1024 MB)
vb.cpus Number of CPU cores available to the VM. In the globals set to 4.

Provider config – Create 2nd disk
• vb.customize to an API to the VboxManage utility of VirtualBox:
– modifyvm: modify vm properties like --clipboard and --draganddrop to
bidirectional
– createmedium: create a new disk:
– --format vdi: Virtual Disk Image
– --variant Standard: dynamically allocated file
– unless File.exists – only create the file if it does not exist yet:
vb.customize [ "createmedium", "disk", "--filename", VM_DISK2, "--format", "vdi",
"--size", VM_DISK2_SIZE , "--variant", "Standard" ]

Provider config – attach2nd disk
• vb.customize to an API to the VboxManage utility of VirtualBox:
– storageattach: attach the disk to the VM
– --storagectl SATA: storage controller to add the disk to
– --port, --device: port and device numbers on the controller
– --type hdd – we’re adding an harddisk here
– --medium: the medium to add
– :id: Special variable that refers to the VM that is created in VirtualBox
vb.customize [ "storageattach", :id , "--storagectl", "SATA", "--port", "2", "--device", "0",
"--type", "hdd", "--medium", VM_DISK2]

Shared/Synchedfolders
• Vagrant by default creates a folder-link or shared folder, a so-called
Synched Folder
• Vagrant project folder (the one with Vagrantfile in it) mapped by
default on /vagrant.
• For stage folder we add:
• Remember:

Provisioning
• Vagrant allows for several provisioners like Puppet, Chef, Ansible, Salt. But a sample
Shell snippet is provided in our Vagrantfile:
• I replaced it with:
• Only main steps,
details in sub scripts
• Script is run as
vagrant
• Can also use external
scripts, even over
URL

Up, up and up it goes…!
• If everything went alright, you’re now ready to fire up your VM.
• Then simply issue the following command in the command window:

Vagrant
1
2
3
4
6
Agenda
7
5

A BIT MOREADVANCEDPROVISIONING

Or actually, the question is:
How do I provision using a provision user
but not have it become the product owner?
SO:…
A bit more advancedprovisioning
Compare Schema Owner
user in database
Compare SYSTEM or
DBA-user in database

Vagrant
A bit more advanced provisioning Provision but not owning
1
2
3
4
6
Agenda
7
5

The provisioning/notowningdilemma
• The provisioning is done using the vagrant user.
• The vagrant user is in the sudoers list -> can run a script using the
permissions of another user;
– sudo su - oracle -c "script“ does not do the trick -> vagrant still owns the files
• It is still the running vagrant user who owns the resulting files and
folders.
• But oracle (or other install owner) is to own it.
• Option: create a new base box with another install user?

Solution:run as another user
• http://man7.org/linux/man-pages/man1/runuser.1.html

Solution:run as another user
• Must be run as root, but that's no problem since vagrant is in the
sudoers list
• Add following to the provisioning block:
• Script installJava.sh left unexplained…
• Database, Fusion Middleware, etc. or other tools requiring other
users can be installed this way

THANKYOU FORYOUR ATTENDANCE, PATIENCE
AND ATTENTION

Q & A

HELPFUL COMMANDS

Some other helpfulcommands
• Basic vagrant commands:
Command Meaning
init Initializes current folder with an initial Vagrantfile
package Packages a VM from provider into a box
up Creates a vm from a box if non-existent, starts up the VM, and
provisions it if not already done
halt Stops the VM
suspend Suspends the VM
destroy Removes the VM

Some other helpfulcommands
• Basic vagrant commands:
Command Meaning
ssh Starts a secure shell connection, logs on as vagrant user
provision Reruns the provisioning of the VM
box add This adds a box with the given address to Vagrant local repo
box list This command lists boxes in Vagrant local repo
box remove This command removes a box from Vagrant local repo
box update Updates a box from an updated base box in local repo

EXAMPLE PROVISION SCRIPTS

Prepare Oracle Linux
• Install required
packages
• Install haveged to
increase entropy
for non-gui VMs
• Add security limits
for oracle user
• Change kernel
settings

Create filesystem
• Create mountpoint
• Create logical
group and volume
using System
Storage Manager
• Add mount point to
fstab

Create Oracle user
• Function to read oracle
user properties from file
• Create groups dba and
oinstall
• Create the user with the
groups
• Set the password from
property file

Create Oracle user
• Add oracle to sudoers
• Create oraInventory
• Create oraInst.loc

20180607 master your vms with vagrant

More Related Content

What's hot

Similar to 20180607 master your vms with vagrant

More from makker_nl

Recently uploaded

20180607 master your vms with vagrant