The document outlines the architecture and prerequisites for deploying OpenShift 4 (OCP4) using User-Provisioned Infrastructure (UPI) on KVM with various configurations for DNS, load balancer, and VM setups. It details the steps for manual configuration and automation using tools like Ansible and Terraform, including network setup, cloud-init, and deployment processes. Additionally, it provides commands and scripts for deployment verification and examples from a demo repository.

1. OCP4 UPI on KVM
BTBS Jay

2. Agenda
- Architecture
- Pre-Requirements
- DNS
- Network
- Load Balancer
- matchbox
- Deploy
- OCP VMs
- Post jobs
- Set registry storage to emptyDir
- Flow
- manual
- jkit
- OCP 4 commands

3. Fedora 28
KVM
LB
Master-
0
Master-1
Master-
2
worker-1
worker-
0
DNS
NetworkManager
api
api-int
.apps
bootstrap
master-0
..
master-n
worker-0
..
worker-n
lb
etcd-0
srv-host
Architecture Network Ranges:
192.168.222.1/24
Cluster Name: upi
Domain Name: example.com
Master: 1
Worker: 1
Docker
matchbox
HTTPD
ignition
kernel
initramfs
rhcos-
bios.raw.gz

4. Pre-requirements(using
NetworkManager)
DNS
NetworkManager
api
api-int
.apps
bootstra
p
master-
0
..
master-n
worker-
0
..
worker-n
lb
etcd-0
srv-host
vi /etc/NetworkManager/dnsmasq.d/libvirt-upi.conf
address=/api.upi.example.com/192.168.222.2
address=/api-int.upi.example.com/192.168.222.2
address=/.apps.upi.example.com/192.168.222.2
address=/bootstrap.upi.example.com/192.168.222.253
address=/master-0.upi.example.com/192.168.222.10
address=/worker-0.upi.example.com/192.168.222.20
address=/lb.upi.example.com/192.168.222.2
address=/matchbox.example.com/192.168.0.184 # hypersvior ip
srv-host=_etcd-server-ssl._tcp.upi.example.com,etcd-0.upi.example.com,2380,0,10
address=/etcd-0.upi.example.com/192.168.222.10
vi /etc/NetworkManager/NetworkManager.conf
dns = dnsmasq

5. DNS Test
api.upi.example.com
api-init.upi.example.com
bootstrap.upi.example.com
master-0.upi.example.com
worker-0.upi.example.com
etcd-0.upi.example.com
matchbox.example.com
lb.upi.example.com
test.apps.upi.example.com

6. Pre-requirements(using Terraforms)
terraform_provider_libvirt.libvirt_network
#network_name= $cluster_name
#network_cidr="192.168.222.0/24"
#network_domain="example.com"
bootp configuration (to point matchbox) ⇐ xlst
resource "libvirt_network" "ocp_network" {
name = "${var.cluster_name}"
mode = "${var.network_mode}"
domain = "${var.cluster_name}.${var.network_domain}"
addresses = ["${var.network_address}"]
bridge = "${var.network_bridge}"
dns = {
enabled = true
local_only = false
}
provisioner "local-exec" {
command = <<EOF
ansible-playbook -i ./ansible/inventory
./ansible/tasks/matchbox_config.yml -e
@ansible/defaults/main.yml
EOF
}
provisioner "local-exec" {
command = <<EOF
ansible-playbook -i ./ansible/inventory
./ansible/tasks/ocp_module.yml -e @ansible/defaults/main.yml
EOF
}
xml {
xslt = "${file("bootp.xsl")}"
}
depends_on = [ "module.matchbox" ]
}

7. Pre-requirements(using Terraforms) - LB
listen ingress-http
bind *:80
mode tcp
option tcplog
option tcp-check
server worker-0 worker-0.upi.example.com:80
check
listen ingress-https
bind *:443
mode tcp
option tcplog
option tcp-check
server worker-0 worker-0.upi.example.com:443
check
listen api
bind *:6443
mode tcp
option tcplog
option tcp-check
server bootstrap
bootstrap.upi.example.com:6443 check
server master-0 master-
0.upi.example.com:6443 check
listen machine-config-server
bind *:22623
mode tcp
option tcplog
option tcp-check
server bootstrap
- VM
- Cloud init
- user-data
- meta-data
- iso(centos)
- Haproxy

8. Pre-requirements(using Terraforms/docker) - Matchbox
Docker
matchbox
ignition
kernel
initramfs
matchbox server container
- /etc/matchbox/server.{crt,key}
- /etc/matchbox/ca.crt
- ~/.matchbox/client.{crt,key}
- /etc/matchbox/ca.crt
Create by
terraform_provider_matchbox
- /var/lib/match/profiles
- /var/lib/match/groups
- /var/lib/match/ignition
- /var/lib/match/assets

9. KVM
Master-0
Master-1
Master-2
worker-1
worker-0
Docker
matchbox
ignition
kernel
initramfs
iPXE
Download
- ignition
- kernel
- initramfs
HTTPD
rhcos-
bios.raw.gz
Pre-requirements(using Terraforms) -
OCP VM

10. Flows
There are many ansible and terraform in places.
However, it is just tools so you can configure everything
manually.
To use this scripts, you can do 2 ways: manual, jkit commands.

11. Full Flows - Init(Initialize)
1. Update config file
2. Create config files (prep/ansible/tasks/generate_config_files.yml)
a. ansible inventory
b. terraform.tfvars
c. connection.tf
d. bootp.xsl
3. Cloud-init (prep/ansible/tasks/cloud_init.yml)
a. user-data
b. meta-data

12. Full Flows - Prep(1)
1. Install packages (prep/ansible/tasks/install_packages.yml)
a. docker
b. httpd
2. DNS (prep/ansible/tasks/dns_config.yml)
3. OCP4 VM Config(prep/ansible/tasks/ocp_vm_config.yml)
a. Create install-config.yaml
b. Generate ignition files
c. Config HTTPD server
4. Deploy Load Balancer
a. terraform init/get/apply

13. Full Flows - Prep(2)
1. Deploy Load Balancer(terraform init/get/apply)
a. Create network
i. Matchbox Server (prep/ansible/tasks/matchbox_config.yml)
1. Generate certs
2. Deploy matchbox container
3. Create matchbox module for matchbox configuration
ii. Config MatchBox
1. groups
2. profiles
3. ignition
iii. OCP module for creating OCP VMs
b. Config HAproxy

14. Full Flows - OCP
1. Deploy OCP4(terraform init/get/apply)
2. Wait for bootstrap-complete
3. Patch imageregistry storage to emptyDir
4. Remove bootstrap from HAproxy backend pool
5. Wait for install-complete

15. Jkit commands
(usage) jkit %CMD% %options%
1. init
2. prep -t {apply(default), dtr}
3. ocp -t {apply(default), dtr}
4. oneshot
5. post
6. update -t {inventory(default), ocp, ocp_module}
7. clean

16. OpenShift 4 Commands
To check bootstrap is completed
- openshift-install --dir ${INSTALL_DIR} wait-for bootstrap-complete
To check OCP installation is completed
- openshift-install --dir ${INSTALL_DIR} wait-for install-complete

17. DEMO
Demo Script
https://github.com/Jooho/jhouse_openshift/blob/master/demos/OCP4/Libvirt/UPI/demo.md