Attention: This presentation contains code examples with small fonts. Please download the slides for better readability.

1. bring your infrastructure under control with
Infrastructor
Stanislav Tiurikov, infrastructor.io, July 2017 ©

2. Introduction
1. Infrastructor is an open source Infrastructure as Code framework
2. It is implemented with Groovy and provides a Domain Specific Language to
describe, manage and provision unix based hosts: either bare-metal servers
and virtual machines
3. Loops, conditions, closures, variables, etc. - whole power of Groovy
Programming Language can be utilized
4. Portable, but requires Java Virtual Machine to run
5. Flexible: there are many ways how to organize your code
6. Unix oriented: currently developed to provision unix based hosts
7. Agentless: only an SSH connectivity is required
8. Has minimal dependencies on the host side, just a few posix utility programs
are used: cat, tee, mkdir, cp, etc.

3. Introduction: an example
inlineInventory { // define an inventory
node host: '10.0.0.10', port: 22, username: 'devops', keyfile: 'devops/private.key'
}.provision { // define a provisioning plan
task('install common packages') { // define a task - a group of actions to run on each inventory node
shell sudo: true, command: 'apt update'
['tmux', 'mc', 'htop'].each {
shell "sudo apt install $it -y"
}
}
}
~ $ infrustructor run -f example.groovy
A provisioning script usually consists of two main parts:
1. An inventory definition - a set of nodes (servers, VMs) you want to provision
2. A provisioning plan definition - which actions to run on the nodes
There is no limitation on inventory and provisioning plan number a single script can
contain
A provisioning script can be executed using infrastructor command line interface:

4. Inventory
There are several ways to define a set of nodes you want to setup:
1. Inline Inventory
2. Aws Inventory
3. Managed Aws Inventory
4. Docker Inventory (for test and play only)

5. Inline Inventory
inlineInventory {
node host: '10.0.0.10', port: 22, username: 'devops', keyfile: 'devops/private.key', tags: [db: true, env: 'test']
node host: '10.0.0.11', port: 22, username: 'devops', keyfile: 'devops/private.key', tags: [db: true, env: 'live']
['10.0.1.10', '10.0.1.11', '10.0.1.12'].each { ip ->
node(host: ip, port: 22) {
username = 'devops'
keyfile = 'devops/private.key'
tags = [service: true, env: 'live']
}
}
}.provision {
// some tasks here
}
Inline Inventory defines a set of nodes in the same file where a provisioning plan is
defined. You can use loops and other Groovy tricks to organize the definition:

6. Aws Inventory
awsInventory(AWS_ACCESS_KEY_ID, AWS_ACCESS_SECRET_KEY, AWS_REGION) {
username = 'devops'
keyfile = 'devops/private.key'
tags = [managed: 'true'] // each AWS instance with tag 'managed:true' will be added to the inventory
usePublicIp = true
}.provision {
task('print all nodes') {
info "$it.id: $it.host" // let’s print all nodes from the inventory
}
}
Aws Inventory can read a set of EC2 instances from AWS and convert it to a set of
nodes. A provisioning plan will be applied to this set of nodes (i.e. EC2 instances):
$ infrastructor run -f aws_example.groovy -v AWS_ACCESS_KEY_ID="XXXXXX" -v AWS_SECRET_KEY_ID="YYYYYY" -v AWS_REGION="eu-central-1"

7. Managed Aws Inventory: EC2 support
managedAwsInventory(AWS_ACCESS_KEY_ID, AWS_SECRET_KEY_ID, AWS_REGION) {
ec2(tags: [managed: true], parallel: 2) { // all EC2 instances with the tag 'managed:true' will be considered as existing nodes
(1..5).each {
node(name: "aws-node-$it") {
imageId = 'ami-12345678'
instanceType = 't2.small'
subnetId = 'sb-12345678'
securityGroupIds = ['sg-00000001', 'sg-00000002']
keyName = 'devops_private_key'
username = 'devops'
keyfile = 'devops/devops.key'
}
}
}
}.provision { ... }
With Managed Aws Inventory we can define a target state of inventory. Infrastructor
will create, update and remove EC2 instances to reach the target state:
$ infrastructor run -f aws_example.groovy -v AWS_ACCESS_KEY_ID="XXXXXX" -v AWS_SECRET_KEY_ID="YYYYYY" -v AWS_REGION="eu-central-1"

8. Managed Aws Inventory: Route53 support
managedAwsInventory(AWS_ACCESS_KEY_ID, AWS_SECRET_KEY_ID, AWS_REGION) {
ec2(tags: [managed: true]) {
node(name: 'web-host-1', tags: ['front-server': true]) { ... }
node(name: 'web-host-2', tags: ['front-server': true]) { ... }
}
route53(hostedZoneId: 'ZZZZZZZZZZZZZZ') {
recordSet(type: 'A', name: 'myservice.infrastructor.io') {
ttl = 600 // seconds
resources = {'front-server:true'} // IPs of instances with tag 'front-server:true' will be added to the record set
usePublicIp = true
}
}
}.provision { ... }
There is an initial support for AWS Route53. Infrasrtuctor can automatically create,
update and remove DNS records in accordance to the current inventory state:
$ infrastructor run -f route53_example.groovy

9. Docker Inventory
def inventory = inlineDockerInventory { // random ports and container ids will be used
node image: 'infrastructor/ubuntu-sshd', tags: [os: 'ubuntu'], username: 'root', password: 'infra'
node image: 'infrastructor/centos-sshd', tags: [os: 'centos'], username: 'devops', password: 'devops'
}.provision {
task(parallel: 2) { shell command: "apt update" }
task(filter: {'os:ubuntu'}) { shell command: "apt install tmux -y" }
task(filter: {'os:centos'}) { shell command: "apt install htop -y" }
}
inventory.shutdown()
Docker Inventory might be useful for some preliminary testing: then you want to
check your provisioning plan quickly:
$ infrastructor run -f docker_inventory_example.groovy

10. Provisioning Plan
inlineInventory { ...
}.provision {
task(name: 'install common packages', tags: {'db:true' || 'web:true'}, parallel: 2) {
shell 'sudo apt install docker.io -y'
}
task(name: 'install MySQL', tags: {'db:true'}) {
shell "sudo docker run -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD mysql:latest"
}
task(name: 'install My Service', tags: {'web:true'}) {
template(source: 'templates/myservice.conf', target: '/etc/myservice.conf') {
bindings = [dbhost: 'storage.internal.infrastructor.io', dbport: 3306, dbuser: 'root', dbpassword: MYSQL_ROOT_PASSWORD]
}
shell 'sudo docker run -d -p 80:80 -v /etc/myservice.conf:/etc/myservice.conf myservice:latest'
}
}.provision { ... }
$ infrastructor run -f provisioning_plan_example.groovy -v MYSQL_ROOT_PASSWORD=XXXXX
A provisioning plan consists of a list of tasks which are executed one by one. It is
possible to define several provisioning plans for the same inventory in a single file:

11. Provisioning Plan: Tasks
inlineInventory {
node id: 'database', host: '10.0.0.10', tags: [db: true, priority: 1], username: 'devops', keyfile: 'devops/private.key'
node id: 'service', host: '10.0.0.11', tags: [service: true, priority: 2], username: 'devops', keyfile: 'devops/private.key'
node id: 'gateway', host: '10.0.0.12', tags: [loadbalancer: true, priority: 1], username: 'devops', keyfile: 'devops/private.key'
...
}.provision {
task(name: 'install common packages', filter: {'db:true' || (!'service:true' || !'priority:2')}, parallel: 2) {
shell 'sudo apt install htop tmux mc -y'
}
...
}
A task definition usually consists of:
1. A human readable task name
2. A filtering expression to define a subset of nodes to run actions on
3. A level of parallelism - amount of nodes to provision in parallel
4. A list of actions to run on the nodes

12. Actions
more info about actions: https://github.com/infrastructor/infrastructor/wiki/Action-Reference
Currently implemented actions:
1. apply - runs a closure within current node context
2. directory - creates a new directory on a remote host
3. fetch - fetches a file from a remote host
4. file - creates a new file with a specified content
5. upload - uploads a file to a remote host
6. group - creates a group on a remote host
7. input - asks for user input: a plain value or a secret
8. insertBlock - inserts a block of text to a file specified on a remote host
9. load - loads a closure from an external file
10. replaceLine - replaces a line in a text file on a remote host
11. shell - runs a shell command on a remote host
12. template - creates a text file based on a specified template
13. user - creates a user on a remote host
14. waitForPort - waits till a specified port of a remote host is ready to accept
connections

13. External configs, runtime parameters and user input
$ infrastructor run -f my_provisioning.groovy -v PARAM1=VALUE1 -v PARAM2=VALUE2 -v PARAM3=VALUE3
inlineInventory { ...
}.provision {
def not_a_secret = input message: 'please, enter a value: ', secret: false
def a_secret = input message: 'please, enter a value: ', secret: true
}
inlineInventory { ...
}.provision {
def configuration = config('configurations/app.config')
info "this is a value from configuration: $configuration.somevalue"
}
more info about external configs: https://github.com/infrastructor/infrastructor/wiki/External-Configuration
Pass parameters via CLI:
Ask user for input during execution:
Use an external file with predefined configuration (will be loaded with Groovy’s ConfigSlurper):

14. Encryption of sensitive data
dbuser=root
dbpassword=mysecrethere
$ infrastructor encrypt -f my_template_with_secret_data.properties -m FULL
1v8sKrlv2J7AKgKUFiFFdsElRL2fYyT6a+oNlq3+jVPnjXJEMMB1W326ltR4
OmAW
$ infrastructor encrypt -f my_secret_data.properties -m PART
dbuser=root
dbpassword=${encrypt('mysecret')}
dbuser=root
dbpassword=${decrypt('AJTwsjM+CbKjmOtV6fD6Mg==')}
more info about encryption: https://github.com/infrastructor/infrastructor/wiki/Encryption-and-Decryption
It is possible to encrypt and decrypt files with sensitive data and then use these files
with actions. In this case the sensitive data it can be safely stored under source
control. There are two supported modes: full (a file will be encrypted completely) and
partial (only specific part of the file will be encrypted). Encryption can be done via
CLI using ‘encrypt’ command as well as decryption with ‘decrypt’ one. AES and
Base64 encoding are used to do encryption.

15. Encryption of sensitive data
inlineInventory { ...
}.provision {
def DECRYPTION_KEY = input message: 'decryption key: ', secret: true
task(name: 'install common packages', tags: {'web:true'}) {
template(source: 'templates/myservice.conf', target: '/etc/myservice.conf') {
decryptionKey = DECRYPTION_KEY
decryptionType = PART // or FULL - depends on how it was done before.
}
shell 'sudo docker run -d -p 80:80 -v /etc/myservice.conf:/etc/myservice.conf myservice:latest'
}
}
more info about encryption: https://github.com/infrastructor/infrastructor/wiki/Encryption-and-Decryption
Decryption is directly supported by ‘upload’ (full mode only) and ‘template’ (both full
and part modes) actions. We can specify a decryption key at runtime:

16. External actions
inlineInventory { ...
}.provision {
def install_common_packages = load 'actions/install_common_pakages.groovy'
task(name: 'install common packages') {
apply closure: install_common_packages
}
}
It is possible to move some actions to an external file to make this logic reusable for
several provisioning scripts. To run such actions we can use a combination of ‘apply’
action and ‘load’ utility function:
install_common_packages.groovy contain a closure:
return {
info "running action on node $node.id" // node variable will be available in this context
shell 'sudo apt update -y'
shell 'sudo apt install tmux htop mc -y'
}

17. Links
1. Infrastructor Project home: https://github.com/infrastructor
2. Apache Groovy Project home: http://groovy-lang.org

18. Thank you!
Stanislav Tiurikov, infrastructor.io, July 2017 ©