What tools are available to install Kubernetes on bare metal. CoreOS, Matchbox, Typhoon, Ignition

1. Kubernetes on metal
Do it Yourself Datacenter
9. Mai 2018
Martin Mailand martin@tuxadero.com

2. Overview
• Slides to explain what I want to show 20 min
• DEMO 30 min
• Q&A 10 min

3. About me
• Datacenter network engineer
• 15 years+ experience
• Currently working at STRATO AG

4. Typhoon
Typhoon

5. Typhoon
• Minimal, stable base Kubernetes distribution
• Declarative infrastructure and configuration
• Free (freedom and cost) and privacy-respecting
• Practical for labs, datacenters, and clouds
• https://github.com/poseidon/typhoon

6. Typhoon
• Kubernetes v1.10.2
• Single or multi-master
• Calico or flannel networking
• TLS-enabled
• RBAC-enabled

7. Typhoon
• AWS
• Bare-Metal
• Digital Ocean
• Google Cloud

8. Typhoon
• Container Linux
• Fedora Atomic

9. Typhoon Config I
cluster_name = "mercury"
matchbox_http_endpoint = "http://192.168.100.11:8080"
container_linux_channel = "alpha"
container_linux_version = "1758.0.0"
cached_install = "true"
kernel_args = ["coreos.autologin=ttyS0"]

10. Typhoon Config II
k8s_domain_name = "node1.example.com"
ssh_authorized_key = "ssh-rsa key"
asset_dir = "/home/martin/terraform/secrets"

11. Typhoon Config III
controller_names = ["node1"]
controller_macs = ["52:54:00:aa:aa:01"]
controller_domains = [„node1.example.com"]
worker_names = [
"node2",
]
worker_macs = [
"52:54:00:aa:aa:02",
]
worker_domains = [
"node2.example.com",
]

12. Toolchain
•Terraform
•Booktube
•Container Linux
•Container Linux Config
•Ignition
•Matchbox

13. Terraform

14. Terraform
• Tool for building, changing, versioning infrastructure
• Can manage popular service provider
• Configuration files describe your application
• Typhoon is a Terraform module

15. Bootkube

16. Bootkube
• tool for launching self-hosted Kubernetes clusters
• deploy a temporary Kubernetes control-plane
• render all of the assets necessary
• TLS assets
• Kubernetes object manifests
• kubeconfig

17. Container Linux

18. Container Linux
• Container Linux is a distro by CoreOS
• Container Linux provides no package manager
• All applications are containers or systemd units
• Immutable - configured during first boot
• Got acquired by RedHat in the beginning of 2018

19. Container Linux
Flatcar Linux
• Fork of Container Linux
• With the approval of CoreOS
• Berlin based
• https://www.flatcar-linux.org/

20. Container Linux Config

21. Container Linux Config
• Human-readable
• YAML
• Cannot be send directly to a Container Linux
provisioning target
• Must be validated
• Transformed into a machine-readable format

22. Container Linux Config
• User
• Storage
• Network
• Systemd units
• Etcd / docker

23. Container Linux Config
passwd:
users:
- name: core
password_hash: "$6$43y3tkl..."
ssh_authorized_keys:
- key1

24. Container Linux Config
storage:
filesystems:
- name: filesystem1
mount:
device: /dev/disk/by-partlabel/ROOT
format: btrfs
wipe_filesystem: true
label: ROOT

25. Container Linux Config
storage:
files:
- path: /opt/file1
filesystem: root
contents:
inline: Hello, world!
mode: 0644
user:
id: 500
group:
id: 501

26. Container Linux Config
systemd:
units:
- name: etcd-member.service
dropins:
- name: conf1.conf
contents: |
[Service]
Environment="ETCD_NAME=infra0"

27. Container Linux Config
Configuration Specification
https://coreos.com/os/docs/latest/configuration.html

28. Config Transpiler

29. Config Transpiler
• Translate
• Validate

30. Container Linux Config Transpiler
Container Linux Config
Config Transpiler
Ignition Config
Provisioned Machine
Ignition

31. Container Linux Config Transpiler
passwd:
users:
- name: core
password_hash: "$6$43y3tkl..."
ssh_authorized_keys:
- key1
{
"ignition": {
"version": "2.1.0"
},
"passwd": {
"users": [
{
"name": "core",
"passwordHash": "$6$43y3tkl...",
"sshAuthorizedKeys": [
"key1"
]
}
]
},
}

32. Ignition

33. Ignition
• only runs once
• runs early in the boot (in the initramfs)
• before the userspace has begun booting
• reads its configuration from a source-of-truth

34. Matchbox

35. Matchbox
• is an HTTP and gRPC service
• renders Ignition configs
• Easily deployed as binary or container

36. Matchbox
• Group
• match machines to profiles based on labels
• MAC address
• UUID
• Stage
• Profile
• set of config templates (e.g. ignition config)

37. Matchbox Group
{
"name": "node1",
"profile": "etcd",
"selector": {
"mac": "52:54:00:89:d8:10"
},
"metadata": {
"fleet_metadata": "role=etcd,name=node1",
"etcd_name": "node1",
"etcd_initial_cluster": "node1=http://node1:2380,
node2=http://node2:2380,node3=http://node3:2380"
}
}

38. Matchbox Profile
{
"id": "etcd",
"name": "Container Linux with etcd2",
"cloud_id": "",
"ignition_id": "etcd.yaml",
"boot": {
"kernel": "/assets/coreos/1576.4.0/coreos_production_pxe.vmlinuz",
"initrd": ["/assets/coreos/1576.4.0/coreos_production_pxe_image.cpio.gz"
"args": [
"coreos.config.url=http://matchbox.foo:8080/ignition?uuid=${uuid}
&mac=${mac:hexhyp}",
"coreos.first_boot=yes",
"coreos.autologin"
]
},
}

39. Recap

40. Recap

41. Demo
DEMO TIME