Test driven Infrastructure development with Ansible and Molecule

Serena Lorenzini, PhD (www.biodec.com)
serena@biodec.com
PloneConf2019
serena@biodec.com Serena Lorenzini, PhD (www.biodec.com) PloneConf2019 1 / 38

Test-driven infrastructure development with Ansible & Molecule
Test-driven infrastructure development with Ansible &
Molecule

What is Ansible?
“Working in IT, you’re likely doing the same tasks over and over. What if
you could solve problems once and then automate your solutions going
forward? Ansible is here to help.”
Ansible is a simple provisioning tool that allows to automate tasks like:
server provisioning
software deployment

Why Ansible?
It is agentless. Everything is done via ssh.
Its YAML configuration files use structure and syntax that will be
familiar to Python programmers.
Ansible’s documentation is vast.
Ansible is easily extended by roles. Roles are Ansibles’ reusable code.
Many roles are available and evaluated by the community (stars,
downloads, comments)
It encourages to discipline yourself to never changing configuration
on your target machines via login.

Why Ansible for Plone?
Here I found the documentation of an Ansible role to provision a Plone
infrastructure.
Installing Plone for production, particularly for a busy or complex site,
is hard and requires you learn about a variety of moving parts:
ZEO server
ZEO clients
Process-control
Load balancing
Reverse-proxy caching
URL rewriting and HTTPS support including certificate management

Provisioning a Plone server
HUGE disclaimer: I don’t know Plone and I have never installed one.
But I like to jump into new things using Ansible when possible.
So, let’s pretend that I had to provision a Plone infrastructure. What
would I do and how would I test my deploy strategy?

Install Ansible
Ansible is a Python package. So, install Ansible using virtualenv and
pip – not your OS package manager
$ apt install virtualenv
$ virtualenv -p /usr/bin/python3.6 py36ansible281
$ source py36ansible281/bin/activate
$ (py36ansible281)$ pip install ansible==2.8.1
Check that Ansible works
$ echo "localhost" > inventory
$ ansible all -m ping -i inventory
$ localhost | success >> {
"changed": false,
"ping": "pong" }
Yes, it is that simple.

The ansible-playbook command
Ping your instance using the ansible-playbook command.
A playbook.yml file:
hosts: all
tasks:
- name: Ping your instance
action: ping
ansible-playbook -i inventory playbook.yml
TASK [Ping] *****************************************
ok: [localhost] => {"changed": false, "ping": "pong"}

Do Ansible the right way
from the beginning: use roles.
Roles provide a directory structure for breaking a (huge) playbook
into multiple files.
This simplifies writing complex playbooks, and it makes them easier
to reuse.
Plus, it makes them easier to be used by others.
Roles can be created by using the ansible-galaxy init command.

Molecule
Instead of using ansible-galaxy I will use Molecule. Molecule is
designed to aid in the development and testing of Ansible roles.
Molecule was adopted by the Ansible project on September 26 2018.
Molecule supports any provider that Ansible supports. Providers can be
bare-metal, virtual, cloud or containers. If Ansible can use it, Molecule
can test it. Molecule simply leverages Ansible’s module system to
manage instances.
So, let’s create a role using Molecule.

Install Molecule
Molecule is a python package, let’s manage it using virtualenv and pip, as
usual.
(py36ansible281)$ pip install molecule==2.22
(py36ansible281)$ pip install testinfra==3.2.0

Develop a role with Molecule
Create a role named example:
$ molecule init role -r example -d docker
--> Initializing new role example...
Initialized role in
/home/serena/Work/test/example successfully.
This command uses ansible-galaxy behind the scenes to generate a new
Ansible role, then it injects a molecule directory in the role, and sets it up
to run builds and test runs in a docker environment. Inside the Molecule
directory is a default directory, indicating the default test scenario.

Now, what’s inside this role scaffold? The usual Ansible directory structure
(tasks, defaults, handlers…), plus a molecule folder.
$ tree -L 1
defaults
handlers
meta
molecule
README.md
tasks
vars

Test-driven infrastructure development with Ansible Molecule
In Ansible everything starts from the tasks/main.yml file:
$ cd example
$ cat tasks/main.yml
There is nothing here. Let’s run some code!
- name: A simple hello
command: echo hello
changed_when: False

Molecule has also created a playbook to run the role,
molecule/default/playbook.yml:
$ ansible-playbook molecule/default/playbook.yml -v
TASK [/home/serena/Work/test/example : A simple hello] ***
ok: [localhost] = {
changed: false,
cmd: [echo hello]],
}
STDOUT: hello

What about tests?
So far we have seen how to:
install Ansible
install Molecule
create the simplest Ansible role following the best practices.
Now, what about tests?

Test using Molecule
In the molecule folder there is a sub folder called default. This is a
Scenario.
A scenario is a test suite for your newly created role.
You can have as many scenarios as you like and Molecule will run one
after the other.
The molecule default Scenario is called “default”
$ cd molecule/default/
$ tree -L 1
.
Dockerfile.j2
INSTALL.rst
molecule.yml
playbook.yml
tests

Test using Molecule
The molecule.yml is for configuring Molecule. It is a YAML file whose
keys represent the high level components that Molecule provides. The
default Molecule Scenario uses Docker.
$ cat molecule.yml
dependency:
name: galaxy
driver:
name: docker
platforms:
- name: instance
image: centos:7
The tests will run in a container, named instance and based onto
the image variable. You can add multiple containers and dedicate parts
of your role to a specific container.

A Molecule test
molecule test --help
Usage: molecule test [OPTIONS]
Test (lint, cleanup, destroy, dependency,
syntax, create, prepare, converge, idempotence,
side_effect, verify, cleanup, destroy).
The linting phases are quite difficult to pass, so, honestly, I usually disable
them. I can re-apply the linting rules once the development phase is
concluded.

A Molecule test
My final molecule.yml:
$ cat molecule/default/molecule.yml
---
lint:
name: yamllint
enabled: False
provisioner:
name: ansible
lint:
name: ansible-lint
enabled: False

A simple Molecule test
The test matrix (not complete):
$ molecule test
-- Validating schema .
Validation completed successfully.
-- Test matrix
default
lint
create
converge
idempotence
destroy

Converge:
-- Scenario: 'default'
-- Action: 'converge'
PLAY [Converge] ************************
TASK [Gathering Facts] ****************
ok: [instance]
TASK [example : A simple hello] *******
ok: [instance]
PLAY RECAP ****************************
instance : ok=2 changed=0

Idempotence:
-- Action: 'idempotence'
Idempotence completed successfully.
Destroy:
-- Action: 'destroy'
PLAY [Destroy] ************************************
TASK [Destroy molecule instance(s)] ****************
changed: [localhost] = (item=instance)
TASK [Delete docker network(s)] ****************
PLAY RECAP *********************************
localhost : ok=2 changed=2serena@biodec.com Serena Lorenzini, PhD (www.biodec.com) PloneConf2019 23 / 38

The Plone Ansible role
Following the Plone documentation there is an Ansible role for provisioning
a Plone infrastructure. Let’s take a look.
$ git clone https://github.com/plone/ansible-playbook.git
$ cd ansible-playbook
$ make all
$ ansible-galaxy -r requirements.yml -p roles install
What’s inside this directory?

There are a lot of directories here, including roles and a few playbooks:
$ tree -L 1
....
README.rst
requirements.yml
roles
sample-medium.yml
sample-multiserver.yml
sample-small.yml
sample-very-small.yml
...

Following the instructions here configure the playbook to include these
roles:
$ cat playbook.yml
...
- name: Include vars from local-configure.yml if found
include_vars: {{ item }}
with_first_found:
- local-configure.yml
...
$ cp sample-very-small.yml local-configure.yml

Following the README I did:
disable munin
configure the admin credentials
run make all to download some other Ansible roles

Test the Plone Ansible role
Now I am ready to inject a molecule test here!
I will use a non default scenario. I will test the Plone deployment using the
ec2 (AWS EC2) driver. (Docker doesn’t work in this case. Services bother
regular Docker containers).
$ molecule init scenario -d ec2
-- Initializing new scenario default...
Initialized scenario in
ansible-playbook/molecule/default successfully.

Using the ec2 driver the default scenario layout looks a bit different:
$ tree -L 1 molecule/default/
molecule/default/
create.yml
destroy.yml
INSTALL.rst
molecule.yml
playbook.yml
prepare.yml
tests

Using the ec2 driver the molecule.yml file looks like this:
driver:
name: ec2
platforms:
- name: instance
image: ami-11bb0e7e
instance_type: t2.micro
...

There is a create.yml file instead of a Dockerfile. This playbook:
creates a security group
creates a keypair and puts it in
.cache/molecule/ansible-playbook/ec2/ssh_key
expects to find the env variable EC2_REGION
expects to find the aws credentials in ~/.aws file
starts the aws instance

The create.yml file:
---
- name: Create
hosts: localhost
connection: local
vars:
ssh_user: admin
ssh_port: 22
security_group_name: molecule
security_group_description: Security group
security_group_rules:
- proto: tcp
from_port: {{ ssh_port }}
to_port: {{ ssh_port }}
cidr_ip: '0.0.0.0/0'
....serena@biodec.com Serena Lorenzini, PhD (www.biodec.com) PloneConf2019 32 / 38

Now change the main playbook that molecule will use:
$ cat molecule/default/playbook.yml
- name: Plone
include: /home/serena/Work/test/ansible-playbook/playbook.ym
To test the Plone Ansible role, which is a collection of ~13 roles.
$ tree -L 1 roles/
roles/
audit
default_config
....
timezone
varnish

I will dare to say that this is becoming a test-driven cloud deployment
with Molecule and Ansible.
Now, that’s a statement!
However, we are getting close to it. Let’s do this.
$ molecule converge
And about ~15 minutes of Ansible later …

The Plone web interface
Figure 1: Plone website

Remeber to destroy this instance:
$ molecule destroy
The molecule test command does the full
converge/idempotence/destroy workflow, but, for keeping things alive for a
bit, I run the molecule converge and molecule destroy commands
one after the other.

Summary
Ansible is easy
Molecule really helps in developing roles and
allows to have many pre-configured test scenarios

The end
Thank you for your attention.

Test driven Infrastructure development with Ansible and Molecule

More Related Content

What's hot

Similar to Test driven Infrastructure development with Ansible and Molecule

Recently uploaded

Test driven Infrastructure development with Ansible and Molecule