Bifrost is a set of playbooks and scripts that install Ironic as a standalone project for use without other OpenStack components. This presentation describes using Ironic with Bifrost playbooks to deploy baremetal hosts across a VPN tunnel.

1. Remote Baremetal
Deployments
Using Bifrost
James Denton – Principal Architect
Rackspace Private Cloud

2. 2
What is it?
To borrow from the Bifrost documentation:
“Bifrost is a set of Ansible playbooks that automates the
task of deploying a base image onto a set of known
hardware using Ironic.
Ironic Bear with sidekick, Bifrost Bear
Bifrost Bear is smaller but just as angry.
• Bifrost consists of 3-major steps:
• Installation
• Enrollment
• Deployment
• Use cases:
• Bootstrapping an environment

3. 3
Know Your Environment
• Baremetal hosts live in DFW environment
• PXE interface connected to local VLAN 450
• Cisco ASA acts as DHCP relay
• IPSec tunnel established between DFW and ORD
• DHCP server lives in ORD environment (across
VPN)
8 - 2x 10G Bond0
9 - Unused
10 - PXE

4. 4
Know Your Environment (con’t)
• Bifrost host is dual-homed:
• front-side interface (vlan 100) for management
• back-side interface (vlan 450) for PXE/DHCP
clients
• Don’t forget the routes:
• Response traffic to PXE/DHCP clients must be
returned via back-end interface
• IPSec tunnel established between ORD and DFW
• Bifrost needs access to OOB IP (iLo/DRAC/BMC)

5. 5
Know Your Environment (Con’t)

6. 6
Installation
Documentation:
• https://docs.openstack.org/bifrost/latest/
Process:
• via bifrost-cli
• via Ansible playbooks
Addresses:
• DFW: 192.168.192.0/25
• DFW DHCP Pool: 192.168.192.32-63
• DFW Static IP: 192.168.192.64-127

7. 7
Installation (Con’t)
Processes:
• ironic
• ironic-inspector
• dnsmasq (dhcp/tftp)
• nginx
• mariadb
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| inspector |
| ironic |
| mysql |
| performance_schema |
| sys |
+--------------------+
6 rows in set (0.001 sec)
ubuntu@ngpc-bifrost-zed:~$ sudo cat /etc/nginx/conf.d/bifrost-httpboot.conf
server {
listen 8080;
server_name ngpc-bifrost-zed;
root /var/lib/ironic/httpboot;
location /var/lib/ironic/httpboot/ {
alias /var/lib/ironic/httpboot/;
}
}
Database:
Nginx:
[pxe]
kernel_append_params = nofb nomodeset systemd.journald.forward_to_console=yes ipa-insecure=1
tftp_server = 192.168.192.239
tftp_root = /var/lib/tftpboot
ipxe_fallback_script = pxelinux.cfg/default
tftp_master_path = /var/lib/ironic/master_images
uefi_ipxe_bootfile_name = ipxe.efi
enable_netboot_fallback = true
[deploy]
http_url = http://192.168.192.239:8080/
http_root = /var/lib/ironic/httpboot
default_boot_option = local
default_boot_mode = uefi
fast_track = True
erase_devices_priority = 0
erase_devices_metadata_priority = 10
[conductor]
automated_clean = true
deploy_kernel = http://192.168.192.239:8080/ipa.kernel
deploy_ramdisk = http://192.168.192.239:8080/ipa.initramfs
rescue_kernel = http://192.168.192.239:8080/ipa.kernel
rescue_ramdisk = http://192.168.192.239:8080/ipa.initramfs
bootloader = http://192.168.192.239:8080/esp.img
[database]
connection = mysql+pymysql://ironic:gDqpAcurJ5QH33GEDRiZ@localhost/ironic?charset=utf8
[inspector]
power_off = False
extra_kernel_params = ipa-inspection-collectors=default,logs
auth_type = http_basic
endpoint_override = http://192.168.192.239:5050
username = admin
password = y87.KX47SGgSArCOQUh0
ironic.conf

8. 8
Installation (Con’t)
Changes:
• Leverage extra dnsmasq configuration and tagging
• /etc/dnsmasq.d/dnsmasq-remote.conf
root@ngpc-bifrost-zed:/etc/dnsmasq.d# cat dnsmasq-remote.conf
log-dhcp
domain-needed
bogus-priv
local=/.lan./
domain=local.lan
dhcp-circuitid=set:dfw,78:72:5d:b9:ea:cf # MAC of local FW PXE_RELAY interface (local to bifrost)
dhcp-circuitid=set:hkg,78:72:5d:b9:ea:cf # MAC of local FW PXE_RELAY interface (local to bifrost)
dhcp-range=dfw,192.168.192.32,192.168.192.63,255.255.255.128,20m # DHCP pool for DFW
dhcp-range=hkg,192.168.193.32,192.168.193.63,255.255.255.128,20m # DHCP Pool for HKG
dhcp-option=dfw,option:router,192.168.192.1 # local GW for DFW nodes
dhcp-option=hkg,option:router,192.168.193.1 # local GW for HKG nodes
dhcp-option=dfw,option:classless-static-route,192.168.192.192/26,192.168.192.1 # get back to bifrost
dhcp-option=hkg,option:classless-static-route,192.168.192.192/26,192.168.193.1 # get back to bifrost
dhcp-option=dfw,option:dns-server,8.8.8.8 # dns
dhcp-option=hkg,option:dns-server,8.8.8.8 # dns

9. 9
Enrollment (Know your Nodes)
Requirements:
• OOB Credentials
• OOB Address
• OOB Method (redfish/ipmi)
• Image location/checksum
storage01-ospcv2-dfw.test.com:
driver_info:
redfish_address: "https://10.15.195.55"
redfish_system_id: "/redfish/v1/Systems/1"
redfish_username: "root"
redfish_password: "JZiOic1"
redfish_verify_ca: false
nics:
- mac: "f4:03:43:57:76:79"
instance_info:
image_source: "http://a43dca281af-7f3f1a.r42.cf1.rackcdn.com/bifrost-ubuntu-jammy-baremetal.qcow2"
image_checksum: "http://a43d4ca28147f3f1a.r42.cf1.rackcdn.com/bifrost-ubuntu-jammy-baremetal.qcow2.md5"
image_type: "partition"
root_gb: "30"
capabilities:
boot_mode: "uefi"
disk_label: "gpt"
driver: "redfish"
inventory_dhcp: true
ipv4_address: "192.168.192.31/25"
ipv4_gateway: "192.168.192.1"
ipv4_nameserver: "9.9.9.9"
properties:
cpu_arch: "x86_64"
root_device: {"hctl": "0:1:0:0"}
name: storage01-ospcv2-dfw.test.com
ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABA<omit>/KMs9ILf4nig3ZKOOFOjz0>
Helpfuls:
• Root device hints
• SSH key
• MAC Address
• IP Address

10. 10
Enrollment (Con’t)
PLAY [Gather facts on localhost] ***********************************************************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************************************************************************
ok: [127.0.0.1]
PLAY [Enroll hardware from inventory into Ironic] ******************************************************************************************************************************
TASK [ironic-enroll-dynamic : Collect facts] ***********************************************************************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-cloud-config : Execute openstack.cloud.config to collect facts] **************************************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-cloud-config : Set openstack_cloud if possible] ******************************************************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-cloud-config : Set openstack.cloud.config auth parameters if not already set.] ***********************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-cloud-config : Set the TLS certificate if present] ***************************************************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-cloud-config : If in noauth mode and no clouds.yaml, unset authentication parameters.] ***************************************************************************
skipping: [storage01-ospcv2-dfw.test.com]
TASK [ironic-enroll-dynamic : Dynamic enrollment] ******************************************************************************************************************************
[WARNING]: Module did not set no_log for skip_update_of_masked_password
changed: [storage01-ospcv2-dfw.test.com -> localhost]
(bifrost) ubuntu@ngpc-bifrost-zed:~/bifrost$ ./bifrost-cli enroll ../finkle-dfw/bifrost-inventory/storage01-ospcv2-
dfw.test.com.yml -e @baremetal-install-env.json
(bifrost) ubuntu@ngpc-bifrost-zed:~/bifrost$ baremetal node list
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+
| 1a903d70-6813-44e3-be9d-8afbdb4a5719 | storage01-ospcv2-dfw.test.com | None | power on | available | False |
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+

11. 11
Deployment
(bifrost) ubuntu@ngpc-bifrost-zed:~/bifrost$ ./bifrost-cli deploy ../finkle-dfw/bifrost-inventory/storage01-ospcv2-dfw.test.com.yml -e @baremetal-install-env.json
PLAY [Gather facts on localhost] ***********************************************************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************************************************************************
ok: [127.0.0.1]
PLAY [Create configuration drive files and deploy machines from inventory] *****************************************************************************************************
...
TASK [bifrost-deploy-nodes-dynamic : Restarting dnsmasq] ***********************************************************************************************************************
changed: [storage01-ospcv2-dfw.test.com -> localhost]
TASK [bifrost-deploy-nodes-dynamic : Collect the checksum of the deployment image.] ********************************************************************************************
skipping: [storage01-ospcv2-dfw.test.com]
TASK [bifrost-deploy-nodes-dynamic : Error if deploy_image_path is not present, and instance_info is not defined] **************************************************************
skipping: [storage01-ospcv2-dfw.test.com]
TASK [bifrost-deploy-nodes-dynamic : Set the calculated checksum] **************************************************************************************************************
skipping: [storage01-ospcv2-dfw.test.com]
TASK [bifrost-deploy-nodes-dynamic : Set generated instance_info] **************************************************************************************************************
skipping: [storage01-ospcv2-dfw.test.com]
TASK [bifrost-deploy-nodes-dynamic : Deploy to hardware] ***********************************************************************************************************************
ok: [storage01-ospcv2-dfw.test.com -> localhost]
PLAY RECAP *********************************************************************************************************************************************************************
127.0.0.1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
storage01-ospcv2-dfw.test.com : ok=24 changed=12 unreachable=0 failed=0 skipped=20 rescued=0 ignored=0
(bifrost) ubuntu@ngpc-bifrost-zed:~/bifrost$ baremetal node list
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+
| 1a903d70-6813-44e3-be9d-8afbdb4a5719 | storage01-ospcv2-dfw.test.com | None | power on | deploying | False |
+--------------------------------------+---------------------------------------------+---------------+-------------+--------------------+-------------+

12. 12
Demo

13. 13
Demo (Con’t)
(bifrost) ubuntu@ngpc-bifrost-zed:~/bifrost$ ssh -i ~/.ssh/ngpc_ospc ubuntu@192.168.192.31
The authenticity of host '192.168.192.31 (192.168.192.31)' can't be established.
ED25519 key fingerprint is SHA256:ZjN7+BPmai4q4AIUiRGrS4pzFHTWDkvE5IlQ7Fczadk.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.192.31' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-72-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Fri Aug 18 20:35:48 UTC 2023
System load: 0.3720703125 Processes: 393
Usage of /: 0.6% of 523.89GB Users logged in: 0
Memory usage: 0% IPv4 address for eno2: 192.168.192.31
Swap usage: 0% IPv4 address for eno2: 192.168.192.96
Temperature: 46.0 C
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@storage01-ospcv2-dfw:~$
ubuntu@storage01-ospcv2-dfw:~$ ip -br a
lo UNKNOWN 127.0.0.1/8 ::1/128
eno1 DOWN
eno2 UP 192.168.192.31/25 192.168.192.96/25 fe80::f603:43ff:fe57:7679/64
eno3 DOWN
eno4 DOWN
eno49 UP fe80::4adf:37ff:fe25:515c/64
eno50 UP fe80::4adf:37ff:fe25:515d/64
ens5f0 UP fe80::4adf:37ff:fe37:b800/64
ens5f1 UP fe80::4adf:37ff:fe37:b801/64

14. 14
Demo (Con’t)
storage01-ospcv2-dfw.test.com:
addresses:
eno2: [ 192.168.192.31/25 ]
host: [ 172.28.232.161/22 ]
mgmt: [ 172.28.236.161/22 ]
storage: [ 172.28.244.161/22 ]
routes:
eno2:
- to: 192.168.192.192/26
via: 192.168.192.1
host:
- to: default
via: 172.28.232.1
(bifrost) ubuntu@ngpc-bifrost-zed:~/netplanner$ ansible-playbook -i simple_inventory site.yml -e @overrides.yml
PLAY [localhost] ************************************************************************************************
TASK [Generate inventory and host_vars] *************************************************************************
TASK [generator : Create host_vars directory] *******************************************************************
ok: [localhost]
TASK [generator : set_fact] *************************************************************************************
ok: [localhost]
TASK [generator : Generate individual host_vars files] **********************************************************
changed: [localhost] => (item=storage01-ospcv2-dfw.test.com)
PLAY [all] ******************************************************************************************************
TASK [Run the netplan!] *****************************************************************************************
TASK [ansible-netplan : Install netplan] ************************************************************************
ok: [storage01-ospcv2-dfw.test.com]
TASK [ansible-netplan : Removing Existing Configurations] *******************************************************
...
TASK [ansible-netplan : Configuring Netplan] ********************************************************************
changed: [storage01-ospcv2-dfw.test.com]
TASK [ansible-netplan : meta] ***********************************************************************************
RUNNING HANDLER [ansible-netplan : Generating Netplan Configuration] ********************************************
changed: [storage01-ospcv2-dfw.test.com]
RUNNING HANDLER [ansible-netplan : Applying Netplan Configuration] **********************************************
changed: [storage01-ospcv2-dfw.test.com]
TASK [Configure hosts] ******************************************************************************************
TASK [configurator : Add local user and add to 'admin' group] ***************************************************
changed: [storage01-ospcv2-dfw.test.com]
TASK [configurator : Disallow SSH password authentication] ******************************************************
changed: [storage01-ospcv2-dfw.test.com]
RUNNING HANDLER [configurator : restart sshd] *******************************************************************
changed: [storage01-ospcv2-dfw.test.com]
PLAY RECAP ******************************************************************************************************
utility01-ospcv2-dfw.test.com : ok=5 changed=0 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
localhost : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
ubuntu@storage01-ospcv2-dfw:~$ ip r
default via 172.28.232.1 dev br-host proto static
172.28.232.0/22 dev br-host proto kernel scope link src 172.28.232.161
172.28.236.0/22 dev br-mgmt proto kernel scope link src 172.28.236.161
172.28.244.0/22 dev br-storage proto kernel scope link src 172.28.244.161
192.168.192.0/25 dev eno2 proto kernel scope link src 192.168.192.31
192.168.192.192/26 via 192.168.192.1 dev eno2 proto static
ubuntu@storage01-ospcv2-dfw:~$ ping 8.8.8.8 -c5
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=1.19 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=1.20 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=1.23 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=118 time=1.14 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=118 time=1.19 ms

15. Rackspace Technology
One Fanatical Place
San Antonio, TX 78218
US sales: 1-800-961-2888
US support: 1-800-961-4454
www.rackspace.com
Copyright © 2021 Rackspace | Rackspace® Fanatical Support® and
other Rackspace marks are either registered service marks or service marks
of Rackspace US, Inc. in the United States and other countries. Features,
benefits and pricing presented depend on system configuration and are
subject to change without notice. Rackspace disclaims any representation,
warranty or other legal commitment regarding its services except for those
expressly stated in a Rackspace services agreement. All other trademarks,
service marks, images, products and brands remain the sole property of
their respective holders and do not imply endorsement or sponsorship.
Thank you