In this talk at the Sydney CoreOS meetup, I took the audience through: a) Installation of CoreOS using VirtualBox and Vagrant b) Items to consider when containerising your platform c) Deploying wordpress across a CoreOS cluster.

1. {CoreOS in anger
Shaun Domingo - @sdomsta
Head of Tech and Operations @ Ninefold

2. The inevitable recap

3. Why CoreOS?
Built-in clustering
Few binaries
No packaging system
Dependent on containers for managing software and apps
Light weight, fat-free
Automatic updates

4.  A single, distributed init system
 Encourages small, ephemeral units / services
 Schedules units into the cluster based on declared
conflicts in systemd unit files
 Fault tolerant
 Can be managed locally or remotely by fleetctl
CoreOS building blocks
systemd
fleet
etcd

5. Orchestration
DIY
Kubernetes (Google)
Mesos + Marathon
(Apache)
CoreOS arsenal
etcd
fleet
flannel
docker
rocket

6. +---------------+ +----------------+ +----------------+
| | | | | |
| core-01 | <> | core-02 | <> | core-03 |
| | etcd | | etcd | |
+---------------+ +----------------+ +----------------+
Exercise 1:
Boot up a 3 machine coreos
cluster

7.  Download & install Vagrant:
 http://www.vagrantup.com/downloads.html
 Download & install Virtualbox:
 https://www.virtualbox.org/wiki/Downloads
1a: install VirtualBox

8.  mkdir ~/coreos; cd ~/coreos
 git clone https://github.com/coreos/coreos-
vagrant.git
1b: Clone the coreos-
vagrant repo

9. cd coreos-vagrant
cp config.rb.sample config.rb
vim config.rb
 change $num_instances:
$num_instances=3
 change $update_channel:
$update_channel=‘beta’
 Change $forwarded_ports:
$forwarded_ports = {8000 => 8888}
1c: modify your config.rb
We’ll use this later
You’ll get 3 CoreOS
VMs
‘alpha’ if you like
bleeding edge,
‘stable’ if you
want to stay sane.

10.  Visit https://discovery.etcd.io/new?size=3
 (you’ll need a new one every time you destroy a
cluster)
 Copy the entire URL provided in the body of
the response
1d: get yourself a
discovery token

11. 4c4,6
< etcd:
---
> update:
> reboot-strategy: etcd-lock
> # etcd:
8,9c10,11
< addr: $public_ipv4:4001
< peer-addr: $public_ipv4:7001
---
> #addr: $public_ipv4:4001
> #peer-addr: $public_ipv4:7001
12c14
< #discovery: https://discovery.etcd.io/<token>
---
> discovery: https://discovery.etcd.io/YOUR_DISCOVERY_TOKEN_HERE
25,26c27,28
< - name: etcd.service
< command: start
---
> #- name: etcd.service
> # command: start
29,30c31,32
< #- name: etcd2.service
< # command: start
---
> - name: etcd2.service
> command: start
1e: modify your user-data
and turn on etcd2
Reboot after first taking a
lock in etcd
Disable etcd (version 1)
Enable etcd2.service,
disable etcd.service
Add your copied
discovery token here

12.  The $private_ipv4 and $public_ipv4 substitutio
n variables are fully supported in cloud-config
on Vagrant. They will map to the first statically
defined private and public networks defined in
the Vagrantfile.
A note on $private_ipv4
and $public_ipv4
substitution variables

13. Examine the vagrant file

14. vagrant up
vagrant status
1f: magic

15. ACHIEVEMENT UNLOCKED!

16. Exercise 2: Get talking to
your cluster

17.  https://github.com/coreos/fleet/releases
 Unzip, add the binary to your path (.bashrc /
.bash_profile / .zshrc)
PATH=$PATH:/path/to/fleetctl
 Restart your terminal
2a: Download fleetctl

18. cd /path/to/coreos-vagrant
eval $(ssh-agent)
# Add the correct vagrant .ssh identity
vagrant ssh-config | sed -n "s/IdentityFile//gp" |
uniq | xargs ssh-add
2b: start your engines
Ensure
ssh-agent
is running
NOTE: You’ll need to rerun these steps each time you blow away your vagrant CoreOS
cluster.

19. export
FLEETCTL_TUNNEL="127.0.0.1:$(vagrant ssh-
config | grep -i 'port' | awk '{print $2; exit}')"
# remove your fleet known hosts
rm ~/.fleetctl/known_hosts
2c: Prepare for takeoff
NOTE: You’ll need to rerun these steps each time you blow away your vagrant CoreOS
cluster.

20. fleetctl list-machines
2d: go go go

21. Need to tweak?
Do it all in the comfort of 127.0.0.1.

22. Exercise 3: Web Scale

23.  Containers, microservices => stateless
 Containers => spin them up quick
 Load balancing => how does it work?
Considerations at scale

24.  When we start a unit, auto-add it to the web
cluster
 When we destroy a unit, auto-remove it from
the web cluster
 When a container is flakey, don’t use it
 Spin up lots of my units without a hassle
Desired

25. +----------------------------+
| |
| Web server / RP |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| App server |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| Database |
| |
+----------------------------+
Standard 3-tier architecture

26. +----------------------------+
| |
| Web server / RP |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| App server |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| Database |
| |
+----------------------------+
… just won’t work here.
Needs to scale,
be fault
tolerant, have
speed and be
nearby.

27. +----------------------------+
| |
| Web server / RP |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| App server |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| Database |
| |
+----------------------------+
… just won’t work here.
Needs to be
stateless, needs
to focus on
dynamic
workloads.

28. +----------------------------+
| |
| Web server / RP |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| App server |
| |
+-------------+--------------+
|
+-------------+--------------+
| |
| Database |
| |
+----------------------------+
… just won’t work here.
Needs to
handle caching,
should really be
a CDN, should
be great at
delivering
static content.

29.  CDN is a must – S3 for asset sync
 Cache as much as you can – varnish,
CloudFront
 Load balancing
 Web server as a RP
 App server / runtime
 Database as a service
 Key/value store – e.g. redis, cache as much as
you can
To achieve true web scale

30.  Tonight’s example:
Wordpress at (localhost) scale
A traditional app, not written with containers and
true scale in mind. Yet it is still incredibly popular
as a blogging and web platform worldwide.
Localhost scale

31. +---------80----------+ +------------+
| | | CDN |<-+
| Load Balancer | +------------+ |
| | |
+--------8888---------+ |
/ | |
/ | |
+-------8888---------+ +--------8888---------+ +-------8888---------+ |
| | | | | | |
| VM1 | | VM2 | | VM3 | |
| | | | | | |
+--------------------+ +---------------------+ +--------------------+ |
| | | | | | |
| Workload balancer | | Workload balancer | | Workload balancer | |
| | | | | | |
+---+---+---+---+---++ +---+---+---+---+---+-+ +--+---+----+---+----+ |
| | | | | | | | | | | | | | | |
+---+---+---+-+-+---+ +--+---+---+---+---+ +-+---+----+---+---+-+ |
| wp| wp| wp| wp| wp| |wp| wp| wp| wp| wp| | wp | wp| wp| wp| wp| ---+
+-------------------+ +------------------+ +--------------------+
Exercise 3 goal

32. +---------80----------+
| |
| Load Balancer |
| |
+--------8888---------+
/ |
/ |
+-------8888---------+ +--------8888--------+ +-------8888---------+
| | | | | |
| core-01 | | core-02 | | core-03 |
| | | | | |
+--------------------+ +--------------------+ +--------------------+
| vulcand | | vulcand | | vulcand |
+--------------------+ +--------------------+ +--------------------+
| discovery sidekick | | discovery sidekick | | discovery sidekick |
+--------------------+---+--------------------+--+--------------------+
| wp wp wp wp wp wpn | | wp wp wp wp wp wpn | | wp wp wp wp wp wpn |
+--------------------+ +--------------------+ +--------------------+
How we’ll solve it
Imagine everything above
the line is solvable with
haproxy … it is.
fleetunits
As wp units are
started, discovery
notifies vulcand
by writing to etcd

33. mkdir ~/coreos; cd ~/coreos
git clone git@github.com:shaundomingo/coreos-
units.git
3a: Clone my repo

34. Modify the following files to add details of your database
server and S3 bucket and AWS keys:
coreos-units/clusterable-
wordpress/wordpress/wordpress@.service
and
coreos-units/clusterable-wordpress/wordpress/wordpress-
admin.service
3b: Add config

35. Either follow clusterable-wordpress/README.md for
instructions on how to run, or if you’re like me and like to
cheat:
cd coreos-units/clusterable-wordpress
./wordpress-up.sh
There’ll be some errors/warnings … that’s normal.
3c: Run *.*

36. 3d: Observe your mighty
cluster firing up

37. fleetctl journal -f discovery@1.service
Watch as it finds wordpress containers and adds locations
into etcd for vulcan.
3e: Check out some logs

38. 3f: Play with your
wordpress site
Go to this URL in your browser:
http://wordpress.local:8888

39.  Our docker units really scale … even if your
local machine doesn’t
Scale

40. Success

41. Demo: vulcand

42.  Built a CoreOS cluster in the complete privacy
of our own laptops… well except for docker, and github, and, OH …
 Brought to life the complexities of container
management and scale
 Made you a CoreOS genius in a few minutes
What have we done?

43. There was a bit of work to
get this going

44.  Unit file automation
 Vulcanproxy persistence
 Monitoring
 Autoscale
 Appc
Where to next?

45.  Let’s talk – you have software, we have … a
great virtualisation layer.
 Steak knives come free
Are you looking to deploy
containers to prod?

46.  @sdomsta
 @ninefold
Follow me, make
me famous
Deploy on Ninefold,
we’ll look after you.
Servers from $7 month.