Edison Wong presented on using Ansible for DevOps automation. He discussed using Ansible to deploy applications to Kubernetes and managing infrastructure as code. He demonstrated testing roles locally with Molecule and testing complex clusters with Vagrant. Wong emphasized starting with test cases and using different test drivers like Docker, LXD, and Vagrant based on the complexity of what is being tested.

1. Hong Kong Open Source
Conference 2019
DevOps with Ansible
From Native to Kubernetes
Edison Wong
2019-06-15

2. Wong Hoi Sing, Edison
●
2005 - Drupal Developer & Contributor
– https://drupal.org/user/33940
●
2008 - HKDUG Co-founder
– https://groups.drupal.org/drupalhk
●
2010 - CEO, PantaRei Design
– hswong3i@pantarei-design.com

7. PantaRei Design
●
Everything Changes and Nothing Remains Still
●
Reinvent Enterprise with Open Source Software and Cloud Computing
●
Hong Kong based FOSS service provider
– Content Management System (CMS) with Drupal
– Cloud Hosting Solution with Amazon Web Services (AWS)
– Team collaborate solution with Atlassian
●
Business Partner with industry leaders
– 2012, AWS Consulting Partner
– 2013, Acquia Partner
– 2013, Atlassian Experts
– 2014, Rackspace Hosting Partner
●
http://pantarei-design.com

9. Outline
●
Why DevOps with Ansible?
●
Hello World with Ansible
●
Ansible Role with Molecule
●
Docker Build with Ansible (!?)
●
Molecule + Vagrant + Ceph + Kubernetes
●
Tips & Tricks
●
Roadmap
●
Q&A

10. Why DevOps with Ansible?
●
HKOSCON 2018
●
SysAdmin Daily Difficulties
●
Why DevOps?
●
Why Ansible?
●
Demo: alvistack-ansible

11. HKOSCON 2018
●
Virtual Hosting with Kubernetes
– Ansible: deployment management
– Docker: application pre-packaging
– CephFS: centralized network storage
– Kubernetes: runtime service lifecycle
management

15. HKOSCON 2018 (cont.)
●
Each talent (i.e. domain) goes into
individual Kubernetes namespace
●
Each namespace have its own Apache,
PHP, MySQL, SSHD, etc
●
All dynamic data store outside
Kubernetes, goes into CephFS

16. SysAdmin Daily Difficulties
●
Different deployment target
●
Test logic before deploy
●
Complex cluster management
●
Documentation
●
No time for learning

17. SysAdmin Daily Difficulties
(cont.)
●
Write-once for all cases
– Native
– Docker/LXD/Vagrant
– AWS/GCE/Azure

19. Why DevOps?
●
Manual install
– Non-repeatable
●
Manual install with document
– Difficult to manage (Docs to Action)
– Always async with production
●
Manual scripting
– Difficult for everything: learn, write, error detection,
debug, etc…

20. Why DevOps? (cont.)
●
DevOps
– Deployment logic as code (i.e. revision
with GIT)
– With error detection and debug tools
– Manage multiple deployment target at
once (e.g. data center, clustering)

21. Why Ansible?
●
Writing “tasks” in YAML
– Human readable == minimize
documentation
– Easy to learn, when compare with
Ruby for Chef or Puppet

22. Why Ansible? (cont.)
A lot of reusable modules
– Simplify complicated logic with error
detection
– Or running “shell” command directly

23. Why Ansible? (cont.)
●
Simple requirement
– Python and Password-less SSH
– Agent-less for managed node

26. Demo: alvistack-ansible
●
https://github.com/alvistack/alvistack-ansibl
e
– Ceph + Kubernetes + Addon
– Support Ubuntu 18.04/16.04 + CentOS 7 +
openSUSE 15.0
– All Roles tested with LXD individually
– Overall Playbook tested with Vagrant
– Simply clone-and-play

28. Hello World with Ansible
●
Install and Setup Ansible
●
Ansible in CLI
●
Ansible in Playbook
●
Ansible in Role
●
Demo: ansible-playbook-ubuntu-desktop

29. Install and Setup Ansible
●
Control Node Requirements
– sudo apt-get install python python-dev openssh-client
– pip install ansible molecule
●
Managed Node Requirements
– sudo apt-get install python-minimal openssh-server
– Password-less SSH login by public-private key pair

30. Ansible in CLI
●
Running command on remote guest is
simple
– ansible -i guest1,guest2, -m ping
– ansible -i guest1,guest2, -m apt -a ‘name=vim
state=present’
– ansible -i guest1,guest2, -m shell -a ‘uname -a’

31. Ansible in Playbook
●
Running multiple “task” once together
●
Finer control than running with CLI
●
Define your inventory then play with it
– ansible-playbook -i inventory/all/hosts
playbooks/setup-everything.yml

34. Ansible in Role
●
Not just “Tasks”, but also:
– Default over-writable variables
– Internal static variables
– Static files for copy
– Template files
– Event handlers
●
A basic build block for complex architecture
– Use Playbook to include different Roles

35. Ansible in Role (cont.)
●
Create a new role with ansible-galaxy
– mkdir ~/.ansible/roles
– cd ~/.ansible/roles
– ansible-galaxy init dummy
●
You could now test it (run via your localhost)
– cd ~/.ansible/roles/dummy
– ansible-playbook -i tests/inventory tests/test.yml

37. Demo: ansible-playbook-
ubuntu-desktop
●
https://github.com/pantarei/ansibl
e-playbook-ubuntu-desktop
– Deployment for Ubuntu Desktop
19.04 + Lot of daily used applications
– Well… Also install `vanilla-gnome-
desktop`…
– Simply Ansible Playbook-only

39. Ansible Role with Molecule
●
Install and Setup Molecule
●
Why Molecule?
●
Multiple Test Cases
●
Multiple Test Drivers
●
Demo: ansible-role-sshd

40. Install and Setup Molecule
●
Molecule could be install with PIP
– pip install ansible molecule
●
Also install with Docker and Vagrant
support
– pip install molecule[docker]
– pip install molecule[vagrant]

41. Why Molecule?
●
DevOps R&D need test
– Role created by ansible-galaxy run via localhost
●
Each test should run in a clean environment
– Role created by molecule run inside
Docker/LXD/Vagrant/etc...
●
Manage test environment(s) is boring
●
Code lint

42. Why Molecule? (cont.)
●
Molecule
– Testing framework for Ansible
– Written in Ansible and Python style
– Write your test case in standard Ansible style
– Manage test environment life-cycle for you
– Code lint
– Idempotence (i.e. run twice to confirm no extra
changes)

43. Why Molecule? (cont.)
●
Create a new Role with molecule
– cd ~/.ansible/roles
– molecule init role -r dummy2 -d docker
– molecule init role -r dummy3 -d lxd
– molecule init role -r dummy4 -d vagrant
●
Now you could run test inside Docker
– cd ~/.ansible/roles/dummy2
– molecule test

46. Multiple Test Cases
●
Molecule support multiple test cases
definition, e.g.
– molecule/centos-7/molecule.yml
– molecule/suse-15/molecule.yml
– molecule/ubuntu-16.04/molecule.yml
– molecule/ubuntu-18.04/molecule.yml

47. Multiple Test Drivers
●
Molecule support multiple drivers
– Docker (default)
– LXD
– Vagrant (i.e. VirtualBox)
– EC2/GCE/Delegated/etc...

51. Demo: ansible-role-sshd
●
https://github.com/alvistack/ansi
ble-role-sshd
– Ansible Role for SSHD installation
– Molecule + LXD + Travis CI
– Support Ubuntu 18.04/16.04 +
CentOS 7 + openSUSE 15.0

54. Docker Build with Ansible (!?)
●
Why NOT Dockerfile?
●
Why NOT ansible-bender?
●
How Dockerfile with Ansible?
●
Demo: docker-jira

55. Why NOT Dockerfile?
●
Back to the origin: why still
custom shell scripting?
– Difficult for everything: learn, write,
error detection, debug, etc…

56. Why NOT ansible-bender?
●
https://github.com/ansible-comm
unity/ansible-bender
– Build Docker Image with standard
Ansible Playbook
– Just need basic Python support
inside target container

57. Why NOT ansible-bender?
(cont.)
●
PROS
– Could integrate with Travis CI
– Push image to DockerHub once build successful
●
CONS
– Can’t trigger auto build from DockerHub if
upstream base image get update (i.e. not
Dockerfile-based)

60. How Dockerfile with Ansible?
●
Kickstart with Dockerfile
●
Install Python, PIP then Ansible
inside Docker image
●
Run Molecule via localhost inside
Docker image (i.e. Delegated)

61. How Dockerfile with Ansible?
●
PROS
– All Roles could be pre-tested
– A simple Playbook via localhost
– Single layer for Docker Image
– Support auto build via DockerHub
●
CONS
– Size bigger than purely Dockerfile (i.e. Python and
Ansible)

63. Demo: docker-jira
●
https://github.com/alvistack/docker-j
ira
– Docker Image packaging for Atlassian
JIRA
– Molecule + Delegated
– All used Roles are LXD tested
– Support DockerHub auto build

66. Molecule + Vagrant + Ceph +
Kubernetes
●
Roles Pre-tested with Molecule
●
Links Roles with GIT Submodule
●
Test Complex Cluster with Vagrant
●
Demo: alvistack-ansible

67. Roles Pre-tested with Molecule
●
Similar as OOP, writing each
Ansible Roles with test cases
●
Simpler for R&D, debug,
maintenance, etc...

68. Links Roles with GIT
Submodule
●
Easy update
●
Easy management

69. Test Complex Cluster with
Vagrant
●
In case of Ceph OSD, truth block device is
required
– Not support file-based loop device
●
In case of Weave, each Kubernetes node
must have unique machine ID
– With LXD all instance get the same host
machine ID

70. Test Complex Cluster with
Vagrant (cont.)
●
Molecule + Docker
– Default, VERY FAST
– NO, (simple) systemd support
– NO, (simple) block device support
– YES, Travis CI support

71. Test Complex Cluster with
Vagrant (cont.)
●
Molecule + LXD
– Fast
– YES, systemd support
– NO, (simple) block device support
– YES, Travis CI support

72. Test Complex Cluster with
Vagrant (cont.)
●
Molecule + Vagrant
– VERY SLOW (i.e. booting VM)
– YES, systemd support
– YES, block device support
– NO, local test only (i.e. VT-x/AMD-v
required)

73. Test Complex Cluster with
Vagrant (cont.)
●
Molecule + LXD for most Ansible
Roles R&D
●
Molecule + Vagrant for final stage
local integration test

74. Demo: alvistack-ansible
●
Fetch source
– git clone https://github.com/alvistack/
alvistack-ansible.git
– cd alvistack-ansible && git submodule
update --init –recursive

75. Demo: alvistack-ansible (cont.)
●
Setup inventory
– cp -rfp inventory/sample inventory/all
– vi inventory/all/hosts

76. Demo: alvistack-ansible (cont.)
●
Run the playbook
– ansible-playbook -i inventory/all/hosts
playbooks/setup-everything.yml

77. Demo: alvistack-ansible (cont.)
●
Manually create Ceph OSD and CephFS, e.g.
– ceph-volume lvm create --blue-store --data /dev/sdc
– ceph osd pool create cephfs_metadata 32 32
– ceph osd pool create cephfs_data 128 128
– ceph fs new cephfs cephfs_metadata cephfs_data

81. Tips & Tricks
●
Always Start with Test Cases
●
Simple Tests Goes Docker
●
Complex Tests Goes LXD
●
Cluster Tests Goes Vagrant
●
Roles Only Handle Single Group
●
Playbook Handle Cross Group
●
Cross Target File Copy with `base64`

82. Roadmap
●
Automatic upgrade plan for each Ansible Role, e.g.
– Save last run schema version (e.g. 1001) to remote with
Local facts (facts.d)
– Fetch and compare with current schema version from
code (e.g. 1004)
– Run the in between upgrade procedures (e.g. 1002.yml,
1003.yml, 1004.yml)
– Update the saved schema version

83. Q&A

84. Contact Us
●
Address: Unit 326, 3/F, Building 16W, No.16
Science Park West Avenue, Hong Kong Science
Park, Shatin, N.T.
●
Phone: +852 3576 3812
●
Fax: +852 3753 3663
●
Email: sales@pantarei-design.com
●
Web: http://pantarei-design.com