GitHub recently announced the end of support for macOS 12, which was the only GitHub-hosted runner capable of supporting nested virtualization. As a result, many teams relying on this feature for CI/CD workflows face the challenge of migrating existing dependencies to alternative solutions. In this session, I shared my experience migrating workloads from GitHub-hosted runners to a cloud-native solution. By leveraging Actions Runner Controller (ARC), KubeVirt, Tekton, and TopoLVM, we were able to provision self-hosted runners on demand, replicating the functionality of nested virtualization while benefiting from the scalability and flexibility of the cloud-native ecosystem.

1. Migrating GitHub Actions
with Nested Virtualization
to Cloud Native Ecosystem
Victor Morales

2. Victor Morales
● +20 yrs as a Software Engineer
● .NET, Java, python, Go
programmer
● OpenStack, OPNFV, ONAP,
Matter, Nephio and CNCF
contributor.

3. Takeaways
It’s possible to run Virtual
Machine instances in a self-hosted
NUC utilizing CNFC projects

4. Context

5. KRD Project
Multi-OS and Multi-VM

6. KRD Project
Nested virtualization

7. GitHub-hosted runners - VM offerings
https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/abo
ut-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

8. New
Event

9. https://github.com/actions/runner-images/issues/10721

10. Solution
Self-hosting GitHub runners
in a Kubernetes cluster

11. Requirements
● Low budget (~250 USD)
● High-VM density (4 CPUs/16 GB RAM/14 GB SSD)
● Kubernetes Cluster (+v1.30)
○ Actions Runner Controller
○ CPU: 1657m requests, 2300m limits
○ Memory: 3500068Ki requests, 6367831296 limits

12. Hardware
Intel Core i7-10710U
● CPU - 10th generation Intel®
Core™ i7-10710U
○ 6 Cores - 12 Total Threads
○ Intel® Virtualization
Technology (VT-x)
● 64 GB DDR4-2666
● Samsung SSD 870 - 1TB
● KINGSTON USNS8154P3256GJ-I
- 256 GB
~ 3 GitHub runners

13. Software
● OS:
○ Ubuntu 22.05 LTS
● All-in-One Cluster:
○ Kubespray v2.27.0
○ Kubernetes v1.30.4
○ Helm v3.16.4
○ Cert-Manager v1.16.2
● Build Golden image:
○ Tekton v0.75.0
○ Container Data Importer v1.61.2
○ Kubevirt Tekton Tasks v0.24.0
● GitHub runners:
○ Action Runner Controller v0.10.1
○ Kubevirt v1.4.0
○ TopoLVM v15.5.0

14. Installation
process

15. ● Disable swap
○ https://github.com/kubernetes/kubernetes/issues/53533
○ sudo swapon —show
● Create Volume Groups
○ sudo vgs
https://github.com/electrocucaracha/krd/blob/master/n
ode.sh
OS configuration
Kubernetes Installation
https://github.com/electrocucaracha/krd/blob/maste
r/_installers.sh#L79-L126

16. Golden image
creation

17. Runner pipeline creation
Pull Ubuntu
cloud image
https://github.com/electrocucaracha/krd/blob/master/resources/ubuntu-runner-pipelineruns.yml
modify-data-object Task
Install GH runner
dependencies
create-vm-from-manifest Task
Wait for VM
completion
wait-for-vmi-status Task
Delete VM
cleanup-vm Task
https://github.com/kubevirt/kubevirt-tekton-tasks
https://github.com/electrocucaracha/krd/blob/master/resources/ubuntu-runner-pipeline.yml

18. CSI implementation of local persistent volumes
https://github.com/topolvm/topolvm
0. Install TopoLVM
$ helm repo add topolvm
https://topolvm.github.io/topolvm
$ helm install --namespace=topolvm-
system topolvm topolvm/topolvm
https://github.com/electrocucaracha/krd/blob/master/_c
hart_installers.sh#L312-L323

19. Allows users to run and manage virtual
machines (VMs) alongside containerized
applications within a Kubernetes environment
https://kubevirt.io/
1. Install Kubevirt
$ kubectl apply -f "
https://github.com/kubevirt/kubevirt/releases/download/${kubevirt_version}/kubevirt-operator.yaml"
$ kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${kubevirt_version}/kubevirt-cr.yaml"
https://github.com/electrocucaracha/krd/blob/master/_installers.sh#L277-L303

20. 1.1 Install Containerized Data Importer
A persistent storage management add-on for Kubernetes, its
primary goal is to provide a declarative way to build Virtual
Machine Disks on PVCs for Kubevirt VMs.
https://kubevirt.io/user-guide/storage/containerized_data_importer/
$ kubectl apply -f "
https://github.com/kubevirt/containerized-data-importer/releases/download/${cdi_version}/cdi-operator.yaml"
$ kubectl apply -f "
https://github.com/kubevirt/containerized-data-importer/releases/download/${cdi_version}/cdi-cr.yaml"
https://github.com/electrocucaracha/krd/blob/master/_installers.sh#L305-L312

21. A framework for creating CI/CD systems
https://tekton.dev/
2. Install Tekton
$ kubectl apply -f "
https://storage.googleapis.com/tekton-releases/operator/previous/$tekton_version/release.yaml"
$ kubectl apply -f "
https://raw.githubusercontent.com/tektoncd/operator/refs/tags/$tekton_version/config/crs/kubernetes/config/$
{KRD_TEKTON_OPERATOR_PROFILE-lite}/operator_v1alpha1_config_cr.yaml
"
https://github.com/electrocucaracha/krd/blob/master/_installers.sh#L354-L366

22. Provides KubeVirt-specific Tekton tasks,which focus on:
● Creating and managing resources (VMs, DataVolumes,
DataSources)
● Executing commands in VMs
● Manipulating disk images with libguestfs tools
● Extracting and uploading disk images from multiple sources
(VM, VMSnapshot, PVC)
https://github.com/kubevirt/kubevirt-tekton-tasks
2.1 Install KubeVirt Tekton Tasks
$ kubectl apply -f "https://github.com/kubevirt/kubevirt-tekton-tasks/releases/download/$(_get_version
kubevirt_tekton_tasks)/kubevirt-tekton-tasks.yaml"

23. GitHub Runner
Execution

24. Kubernetes operator that orchestrates and scales self-hosted
runners for GitHub Actions.
https://github.com/actions/actions-runner-controller/
3. Install Actions Runner Controller
$ helm install arc --namespace arc-systems oci://ghcr.io/actions/actions-runner-
controller-charts/gha-runner-scale-set-controller

25. Runner scale sets are a group of homogeneous runners that can be assigned jobs from GitHub.
3.1 Install Runner Scale set
$ export GITHUB_PAT="<PAT>"
$ helm install vm-self-hosted --namespace krd
--set githubConfigUrl=https://github.com/electrocucaracha/krd
--set githubConfigSecret.github_token="${GITHUB_PAT}"
oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

26. https://github.com/electrocucaracha/krd/blob/master/resources/kubevirt-runner/vm.yml
VM runner template

27. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
ubuntu-jammy-dv datavolume
ubuntu-jammy-vm virtualmachine

28. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
vm-self-hosted-xxx-runner-xxx ephemeralrunner
ubuntu-jammy-dv datavolume
ubuntu-jammy-vm virtualmachine

29. https://github.com/electrocucaracha/krd/blob/master/helm/arc/ubuntu-jammy-values.yml

30. Create VMI
Resources
runner.CreateResources()
Wait for Virtual
Machine Instance
runner.WaitForVirtualMachineIn
stance()
Delete VMI
Resource
runner.DeleteResources()
kubevirt-actions-runner
https://github.com/electrocucaracha/kubevirt-actions-runner/blob/master/cmd/kar/app/root.go#L44-L58

31. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
vm-self-hosted-xxx-runner-xxx pod
vm-self-hosted-xxx-runner-xxx ephemeralrunner
ubuntu-jammy-dv datavolume
ubuntu-jammy-vm virtualmachine

32. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
vm-self-hosted-xxx-runner-xxx pod
virt-launcher-vm-self-hosted-xxx-runner pod
vm-self-hosted-xxx-runner-xxx ephemeralrunner
ubuntu-jammy-dv datavolume
ubuntu-jammy-vm virtualmachine

33. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
vm-self-hosted-xxx-runner-xxx pod
virt-launcher-vm-self-hosted-xxx-runner pod
vm-self-hosted-xxx-runner-xxx ephemeralrunner
ubuntu-jammy-dv datavolume
ubuntu-jammy-dv-vm-self-hosted-xxx-runner-xx
datavolume
ubuntu-jammy-vm virtualmachine

34. K8s cluster
arc-system krd
arc-gha-rs-controller-xxx pod
AutoScalingListener Controller
AutoScalingRunnerSet Controller
EphemeralRunner Controller
EphemeralRunnerSet Controller
vm-self-hosted-xxx-listener pod
vm-self-hosted-xxx-runner-xxx pod
virt-launcher-vm-self-hosted-xxx-runner pod
vm-self-hosted-xxx-runner-xxx ephemeralrunner
ubuntu-jammy-dv datavolume
ubuntu-jammy-dv-vm-self-hosted-xxx-runner-xx
datavolume
vm-self-hosted-xxx-runner-xx
virtualmachineinstance
ubuntu-jammy-vm virtualmachine

36. Q&A

37. Backup

38. Garbage collectors
https://github.com/electrocucaracha/krd/blob/master/resources/arc-cleanup.yml
Kubernetes Cron Jobs created to clean up leftovers
● ephemeralrunners-garbage-collector
● succeeded-vmis-garbage-collector
● failed-vmis-garbage-collector