The challenges brought to the cloud native community are ever expanding. Luckily also the tools and the hardware support is expending.
Edge is a topic which appears ever more often. The challenges there are as much as the use cases. Here we want to explore how to build a true portable datacenter what can travel easily and brings enough power to support certain applications. A datacenter which fits in a suitcase and is possible to bring to remote locations to help there to bring computing power directly to the place to be.
With the help of various CNCF projects, this comes to live. The base is a MiniITX Clusterboard with multiple ARM based hardware nodes. On top the utilization will take place with KubeVirt, and we utilize the small control plane approach of the Kubermatic Kubernetes platform to create a real multi cluster Datacenter. Follow me in this Case Study to unlock a really portable Datacenter.
This case study and the open source approach of its design is aimed to give a possible solution for small real edge datacenters. An easy to built public available blueprint for a portable datacenter shows the strong parts of our Open Source community. The utilization of multiple CNCF open source projects and the sharing of the created blueprint to everyone.
So it will be easy for more people to explore the edge world by themselves and can create their on solutions based on it
3. Why would you need a datacenter in a
suitcase?
3
Whats the fuzz all about
4. 4
The Idea
A portable ARM processor-based data center that
leverages the benefits of Kubernetes and uses KubeVirt
as a hypervisor to make the setup dynamic.
5. 5
ARM Processors
ARM (Advanced RISC Machines) is a family of RISC
architectures (Reduced Instruction Set Computing) for
embedded systems. ARM processors are known for their
low power consumption and high performance
6. 6
Benefits
✘ Power Efficiency
✘ Cost
✘ Diverse Ecosystem
✘ Scalability
✘ Hardware Customization
Benefits and Downsites
Downsites
✘ Software Compatibility
✘ Performance Per Core:
✘ Standardization Issues
✘ Migration Efforts
✘ Less Mature in Server Space
✘ Vendor Lock-in Concerns:
7. 7
TuringPI
"We needed a board for multiple ARM nodes - in this
example, we use the TuringPI 4-node mini ITX board with
a built-in Ethernet switch.
But in reality, any other board/setup where you can link
ARM processors together will also work.
8. 8
Specs
❏ 1 GBE L2 Switch
❏ Baseboard Management Controller
❏ 2x 1GbE ports
❏ 2x Mini PCI Express Slots
❏ 1x DSI
❏ 4x USB 3.0 1x USB 2.0
❏ RTC battery slot
❏ VLAN Support
❏ 2x SATA 3 Ports 6 Gps
❏ 4x M.2 ports 2260 2280 Key-M NVMe support
❏ 1x HDMI
❏ 1x 40-pin GPIO Rasperry Pi compatible
❏ SIM slot
❏ Power Supply 24-pin ATX
❏ Mix multiple compute Modules
9. 9
It was expensive, wasn’t it?
1x Turing Pi = €249
4x CM4 Raspberry Pi = €500
4x Adapter Turing Pi Raspberry Pi = €40
2x SSD ~ €100
2x SATA Cable = €10
1x 500W Power Supply = €30
1x USB 2.0 A to A Cable = €7
= €936
2 TB Storage
16 Cores
32 GB Memory
13. 13
Have you heard about your Savior KubeVirt?
https://kubevirt.io/
run and manage virtual machines alongside your container workloads
✘ Combine existing virtualized workloads with new container
workloads
✘ Leverage KubeVirt and Kubernetes to manage virtual machines
✘ Have Container Workload and traditional workloads side by side
✘ Is not a hypervisor 😉
14. KubeOne - yeah sorry some self
advertisement
14
First we Need a Cluster
15. 15
What and Why is KubeOne
https://github.com/kubermatic/kubeone
KubeOne helps you to create update and manage a Kubernetes
Cluster
✘ 100% Open Source
✘ Declarative style
✘ Vanilla Kubernetes and easily extendable
✘ Full IaC approach - even machine creation is possible (we do not
need)
17. Migrate from the legacy package repositories (apt.kubernetes.io and
yum.kubernetes.io) to the Kubernetes community-hosted package
repositories (pkgs.k8s.io). The legacy repositories have been deprecated
as of August 31, 2023 and will be frozen starting from September 13,
2023
17
SERVICE Announcement
18. 18
Back to Topic ähh Virt - ARM meets KubeVirt
✘ Comprehensive ARM integration
✘ Support for various platforms like
Raspberry Pi, NVIDIA Jetson, and
Qualcomm Snapdragon
✘ Wide support for different OS
Ubuntu, Fedora, and Alpine
✘ KubeVirt API and CLI can be used
to manage both ARM and
x86-based VMs
✘ Less performance than x86-based
hardware
✘ Documentation not super good yet
19. 19
Maybe to much Overhead?
✘ Control Plane Overhead - KubeVirt controller and API server, will consume some
CPU resources
✘ Virtual Machine Overhead - similar to running VMs on traditional platforms
✘ Libvirt and QEMU: KubeVirt uses libvirt and QEMU for virtualization - overhead for
virtualization process itself
✘ S
✘ Overcommitment possible -KubeVirt allows for CPU overcommitment
✘ Resource Allocation - control the CPU resources allocated to KubeVirt
components and VMs
21. 21
Theory is cool - doing is shitty
✘ Easy fixes: using the right USB Cable
✘ Hard things to debug: Missing packages for PI & ARM
✘ Stupid things: buying RaspberryPI that have eMMC storage and then wondering
why ssd is not supported
✘ Using an Ubuntu that have not a supported linux modules for pi
✘ Using the wrong network
✘ Patching the firmware before presentations and fucking it up
23. 23
Going further
✘ Multi Cluster deployment - more overhead - mor flexibility e.g. KKP
✘ Include portable power source
✘ Change Pi to other computing units like TPM or similar to utilize nvme slots for all
nodes
✘ Built an actual case around it
✘ Do some real stress tests to the system
✘ Find an automated installation way utilizing pxeboot and tinkerbell maybe
24. 24
Use Cases
Expedition /
Research
Have a datacenter
on expeditions to
analyze found data
right on the spot -
lightweight
carryable and
possible to share
data
Small Sites
Shops with limited
space - trains -
cars
Trucks, anything
that has limited
space
Military
Portable
Datacenters that
can be operated
by single forces
and is highly
mobile and easy to
destroy if needed
