High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

M
Mathias Renner
••
Professorship for Computer Science Communication Services, Telecommunication Systems and Computer Networks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster HighLoad++ Conf, Moskow | Docker Meetup, Bamberg November 8, 2016 | November 17, 2016 @MathiasRenner | Hypriot.com
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Content 1 Motivation 2 Why Kubernetes and Docker? 3 Related Work 4 Test Setup 5 Testing Scenarios 6 Results 7 Live Demo 8 Outlook 9 Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 2/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular Own contribution for Docker on ARM High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch OpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. This work’s contribution: Extend tests of Nissen/Jensen and compare K8s against Docker Swarm. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Hardware Setup with PicoCluster Figure 1: Pico Cluster 5xRPi 3, https://www.picocluster.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 6/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... Software Which OS? Raspbian vs. HypriotOS vs. ArchLinux Flashing (I stopped counting), device-init... Network: 2 interfaces per device, check DHCPs ... Update Software Monitoring system that supports all required capabilities Configure Desktop machine: No network interference, appropriate software to manage the cluster (terminator) Setup NTP SSH listen on two interfaces Setup Swapspace Automate, automate, automate! ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Docker Easy since included in HypriotOS. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 8/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install Recently, setup became as easy as with Docker. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software stack for experiments Kubernetes Stack Docker Swarm Stack rpicloud/greeting-rpi rpicloud/greeting-rpi K8s@ARM Docker Docker HypriotOS HypriotOS High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 10/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test Resilience Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resource Usage Table 1: First results of resource consumption measurements Sc. CPU usage, % Mem usage, % Proc CPU D., % Proc mem D., MB 1 0.44 4.21 0.05 2.47 2 1.43 / 0.83 7.87 / 7.4 1.1 / 0.44 57.45 / 2.42 3 14.74 / 2.01 34.21 / 12.68 5.76 / 1.31 479.51 / 160.77 Legend: Scenario 1: Fresh install, monit and Docker daemon running Scenario 2: After docker init Scenario 3: After starting K8s Also, in Scenario 3, Docker created 41 process children. In Scenario 1 & 2, Docker has 1. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 12/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned And then the Big Bang... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 13/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 → Both tools were basically not working to the minimum extent. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 2: Timeout of curl command against the cluster High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 15/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 3: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 16/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 4: Incoming network traffic Figure 5: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 17/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 6: One of the resolved routing issues for Docker 1.12.2 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 18/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Plan B... Throw most of the effort away and figure out a Plan B... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 19/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Final Software stack of experiments Kubernetes Stack Docker Swarm Stack (Nissen/Jensen) rpicloud/greeting-rpi rpicloud/greeting-rpi K8s v1.2 + K8s@ARM v0.7 Docker v1.11.x Docker v1.12.3 ArchLinux ARM latest HypriotOS v1.1.0 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 20/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 7: Breakpoints of load test with Kubernetes Test has performed with 10.000 req at eatch step. Stepsize=100 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 21/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 8: Breakpoints of load test with Docker Swarm Mode Note that the blue and green line overlap very much. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 22/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Load Test Table 2: Breakpoints of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 ca. 900 reqests/seconds ca. 600 reqests/seconds 5 ca. 1400 reqests/seconds ca. 900 reqests/seconds 10 ca. 1500 reqests/seconds ca. 900 reqests/seconds High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 23/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 9: Successful requests onto the Kubernetes cluster with 1 replica Kubernetes: Total outage of 47 seconds Docker Swarm Mode: Total outage of 72 seconds (+53%) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 24/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 10: Successful requests onto the cluster with 2 replica High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 25/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resilience Test Table 3: Success rates of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 74.65% 61.74% 2 99.87% 99.85% 5 99.98% 99.93% High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 26/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Live Demo: ”Let’s crash the cluster!” High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 27/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level Use single board computers with higher network I/O High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Questions ? @MathiasRenner www.Hypriot.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 30/30
1 of 72

High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

••
Download to read offline
Software

The new Docker Swarm Mode claims to ship all features required to make a service in a cluster highly available. Thus, next to Kubernetes, Docker could be the second reasonable orchestration tool available that can be deployed on a Raspberry Pi cluster. But let's not trust the release notes – let's see how the reality looks like! Which of the two will win?

M
Mathias Renner

Recommended

Evaluation of High Availability Performance of Kubernetes and Docker Swarm on... by
Evaluation of High Availability Performance of Kubernetes and Docker Swarm on...Evaluation of High Availability Performance of Kubernetes and Docker Swarm on...
Evaluation of High Availability Performance of Kubernetes and Docker Swarm on...Ontico
2.4K views•85 slides
Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020) by
Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)
Canary Releases on Kubernetes with Spinnaker, Istio, & Prometheus (2020)Kublr
213 views•24 slides
How to Run Kubernetes in Restrictive Environments by
How to Run Kubernetes in Restrictive EnvironmentsHow to Run Kubernetes in Restrictive Environments
How to Run Kubernetes in Restrictive EnvironmentsKublr
133 views•26 slides
Building Portable Applications with Kubernetes by
Building Portable Applications with KubernetesBuilding Portable Applications with Kubernetes
Building Portable Applications with KubernetesKublr
168 views•28 slides
Portable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins by
Portable CI/CD Environment as Code with Kubernetes, Kublr and JenkinsPortable CI/CD Environment as Code with Kubernetes, Kublr and Jenkins
Portable CI/CD Environment as Code with Kubernetes, Kublr and JenkinsKublr
145 views•27 slides
GitOps - Operation By Pull Request by
GitOps - Operation By Pull RequestGitOps - Operation By Pull Request
GitOps - Operation By Pull RequestKasper Nissen
1.4K views•85 slides

More Related Content

What's hot

Defining & Enforcing Policies the GitOps Way by
Defining & Enforcing Policies the GitOps WayDefining & Enforcing Policies the GitOps Way
Defining & Enforcing Policies the GitOps WayWeaveworks
655 views•22 slides
Modern DevOps with Spinnaker - Olga Kundzich by
Modern DevOps with Spinnaker - Olga KundzichModern DevOps with Spinnaker - Olga Kundzich
Modern DevOps with Spinnaker - Olga KundzichVMware Tanzu
1.3K views•59 slides
Zero-downtime deployment of Micro-services with Kubernetes by
Zero-downtime deployment of Micro-services with KubernetesZero-downtime deployment of Micro-services with Kubernetes
Zero-downtime deployment of Micro-services with KubernetesWojciech Barczyński
411 views•81 slides
Distributed tensorflow on kubernetes by
Distributed tensorflow on kubernetesDistributed tensorflow on kubernetes
Distributed tensorflow on kubernetesinwin stack
1.6K views•20 slides
從Google cloud看kubernetes服務 by
從Google cloud看kubernetes服務從Google cloud看kubernetes服務
從Google cloud看kubernetes服務inwin stack
788 views•52 slides
Hystrix by
HystrixHystrix
HystrixDiego Pacheco
2.1K views•24 slides

What's hot(20)

Defining & Enforcing Policies the GitOps Way by Weaveworks
Defining & Enforcing Policies the GitOps WayDefining & Enforcing Policies the GitOps Way
Defining & Enforcing Policies the GitOps Way
Weaveworks•655 views
Modern DevOps with Spinnaker - Olga Kundzich by VMware Tanzu
Modern DevOps with Spinnaker - Olga KundzichModern DevOps with Spinnaker - Olga Kundzich
Modern DevOps with Spinnaker - Olga Kundzich
VMware Tanzu•1.3K views
Zero-downtime deployment of Micro-services with Kubernetes by Wojciech Barczyński
Zero-downtime deployment of Micro-services with KubernetesZero-downtime deployment of Micro-services with Kubernetes
Zero-downtime deployment of Micro-services with Kubernetes
Wojciech Barczyński•411 views
Distributed tensorflow on kubernetes by inwin stack
Distributed tensorflow on kubernetesDistributed tensorflow on kubernetes
Distributed tensorflow on kubernetes
inwin stack•1.6K views
從Google cloud看kubernetes服務 by inwin stack
從Google cloud看kubernetes服務從Google cloud看kubernetes服務
從Google cloud看kubernetes服務
inwin stack•788 views
Hystrix by Diego Pacheco
HystrixHystrix
Hystrix
Diego Pacheco•2.1K views
Kubernetes community demo march 16 2017 by rhirschfeld
Kubernetes community demo march 16 2017Kubernetes community demo march 16 2017
Kubernetes community demo march 16 2017
rhirschfeld•374 views
Deploying to Day N Operations of Kubernetes and Containerized Apps by Nebulaworks
Deploying to Day N Operations of Kubernetes and Containerized AppsDeploying to Day N Operations of Kubernetes and Containerized Apps
Deploying to Day N Operations of Kubernetes and Containerized Apps
Nebulaworks•72 views
Should developers care about dockerfiles and kubernetes resources by Kasper Nissen
Should developers care about dockerfiles and kubernetes resourcesShould developers care about dockerfiles and kubernetes resources
Should developers care about dockerfiles and kubernetes resources
Kasper Nissen•194 views
How to integrate Kubernetes in OpenStack: You need to know these project by inwin stack
How to integrate Kubernetes in OpenStack: You need to know these projectHow to integrate Kubernetes in OpenStack: You need to know these project
How to integrate Kubernetes in OpenStack: You need to know these project
inwin stack•842 views
Kubernetes in kubernetes 搭建高可用環境 by inwin stack
Kubernetes in kubernetes 搭建高可用環境Kubernetes in kubernetes 搭建高可用環境
Kubernetes in kubernetes 搭建高可用環境
inwin stack•1.1K views
Kubernetes service with ha by Sam Zheng
Kubernetes service with haKubernetes service with ha
Kubernetes service with ha
Sam Zheng•427 views
K8s by Sam Zheng
K8sK8s
K8s
Sam Zheng•430 views
Zero downtime deployment of micro-services with Kubernetes by Wojciech Barczyński
Zero downtime deployment of micro-services with KubernetesZero downtime deployment of micro-services with Kubernetes
Zero downtime deployment of micro-services with Kubernetes
Wojciech Barczyński•599 views
Distributed tensorflow on kubernetes by inwin stack
Distributed tensorflow on kubernetesDistributed tensorflow on kubernetes
Distributed tensorflow on kubernetes
inwin stack•645 views
Introducing Pico - A Deep Learning Platform using Docker & IoT - Sangam Biradar by sangam biradar
Introducing Pico - A Deep Learning Platform using Docker & IoT - Sangam BiradarIntroducing Pico - A Deep Learning Platform using Docker & IoT - Sangam Biradar
Introducing Pico - A Deep Learning Platform using Docker & IoT - Sangam Biradar
sangam biradar•444 views
Using source code management patterns to configure and secure your Kubernetes... by Giovanni Galloro
Using source code management patterns to configure and secure your Kubernetes...Using source code management patterns to configure and secure your Kubernetes...
Using source code management patterns to configure and secure your Kubernetes...
Giovanni Galloro•103 views
Unlimited Staging Environments by Codefresh
Unlimited Staging EnvironmentsUnlimited Staging Environments
Unlimited Staging Environments
Codefresh•2.8K views
DevOps with Azure, Kubernetes, and Helm Webinar by Codefresh
DevOps with Azure, Kubernetes, and Helm WebinarDevOps with Azure, Kubernetes, and Helm Webinar
DevOps with Azure, Kubernetes, and Helm Webinar
Codefresh•2.2K views
Read ebook Kubernetes Cookbook: Building Cloud Native Applications Full Access by illingstabilityb
Read ebook Kubernetes Cookbook: Building Cloud Native Applications Full AccessRead ebook Kubernetes Cookbook: Building Cloud Native Applications Full Access
Read ebook Kubernetes Cookbook: Building Cloud Native Applications Full Access
illingstabilityb•40 views

Similar to High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

Cloud-Native Operations with Kubernetes and CI/CD by
Cloud-Native Operations with Kubernetes and CI/CDCloud-Native Operations with Kubernetes and CI/CD
Cloud-Native Operations with Kubernetes and CI/CDVMware Tanzu
1.9K views•80 slides
Kubernetes in Highly Restrictive Environments by
Kubernetes in Highly Restrictive EnvironmentsKubernetes in Highly Restrictive Environments
Kubernetes in Highly Restrictive EnvironmentsKublr
358 views•25 slides
Operationalizing Amazon EKS by
Operationalizing Amazon EKSOperationalizing Amazon EKS
Operationalizing Amazon EKSJim Bugwadia
444 views•37 slides
A Primer on Kubernetes and Google Container Engine by
A Primer on Kubernetes and Google Container EngineA Primer on Kubernetes and Google Container Engine
A Primer on Kubernetes and Google Container EngineRightScale
4.5K views•61 slides
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o... by
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...NETWAYS
106 views•51 slides
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De... by
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...NETWAYS
17 views•51 slides

Similar to High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster(20)

Cloud-Native Operations with Kubernetes and CI/CD by VMware Tanzu
Cloud-Native Operations with Kubernetes and CI/CDCloud-Native Operations with Kubernetes and CI/CD
Cloud-Native Operations with Kubernetes and CI/CD
VMware Tanzu•1.9K views
Kubernetes in Highly Restrictive Environments by Kublr
Kubernetes in Highly Restrictive EnvironmentsKubernetes in Highly Restrictive Environments
Kubernetes in Highly Restrictive Environments
Kublr•358 views
Operationalizing Amazon EKS by Jim Bugwadia
Operationalizing Amazon EKSOperationalizing Amazon EKS
Operationalizing Amazon EKS
Jim Bugwadia•444 views
A Primer on Kubernetes and Google Container Engine by RightScale
A Primer on Kubernetes and Google Container EngineA Primer on Kubernetes and Google Container Engine
A Primer on Kubernetes and Google Container Engine
RightScale•4.5K views
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o... by NETWAYS
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...
OSDC 2017 - Timo Derstappen - Automating kubernetes cluster operations with o...
NETWAYS•106 views
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De... by NETWAYS
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...
OSDC 2017: Automating Kubernetes Cluster Operations with Operators by Timo De...
NETWAYS•17 views
Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ... by MayaData Inc
Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ... Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ...
Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ...
MayaData Inc•246 views
Persistent Storage for stateful applications on Kubernetes made easy with Ope... by MayaData Inc
Persistent Storage for stateful applications on Kubernetes made easy with Ope...Persistent Storage for stateful applications on Kubernetes made easy with Ope...
Persistent Storage for stateful applications on Kubernetes made easy with Ope...
MayaData Inc•447 views
K8s architecture meetup2- k8saraby by Walid Shaari
K8s architecture meetup2- k8sarabyK8s architecture meetup2- k8saraby
K8s architecture meetup2- k8saraby
Walid Shaari•560 views
Kubernetes for Java developers by Robert Barr
Kubernetes for Java developersKubernetes for Java developers
Kubernetes for Java developers
Robert Barr•170 views
Building a Kubernetes cluster for a large organisation 101 by Ed Schouten
Building a Kubernetes cluster for a large organisation 101Building a Kubernetes cluster for a large organisation 101
Building a Kubernetes cluster for a large organisation 101
Ed Schouten•254 views
Kubernetes extensibility by Docker, Inc.
Kubernetes extensibilityKubernetes extensibility
Kubernetes extensibility
Docker, Inc.•1K views
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware by VMUG IT
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
VMUG IT•262 views
Kubernetes 101 Workshop by Bret McGowen - NYC Google Developer Advocate
Kubernetes 101 WorkshopKubernetes 101 Workshop
Kubernetes 101 Workshop
Bret McGowen - NYC Google Developer Advocate•1.9K views
Federated Kubernetes: As a Platform for Distributed Scientific Computing by Bob Killen
Federated Kubernetes: As a Platform for Distributed Scientific ComputingFederated Kubernetes: As a Platform for Distributed Scientific Computing
Federated Kubernetes: As a Platform for Distributed Scientific Computing
Bob Killen•1.5K views
Reliable Performance at Scale with Apache Spark on Kubernetes by Databricks
Reliable Performance at Scale with Apache Spark on KubernetesReliable Performance at Scale with Apache Spark on Kubernetes
Reliable Performance at Scale with Apache Spark on Kubernetes
Databricks•2.2K views
DevOps with Kubernetes and Helm - Jenkins World Edition by Jessica Deen
DevOps with Kubernetes and Helm - Jenkins World EditionDevOps with Kubernetes and Helm - Jenkins World Edition
DevOps with Kubernetes and Helm - Jenkins World Edition
Jessica Deen•262 views
Why Kubernetes? Cloud Native and Developer Experience at Zalando - Enterprise... by Henning Jacobs
Why Kubernetes? Cloud Native and Developer Experience at Zalando - Enterprise...Why Kubernetes? Cloud Native and Developer Experience at Zalando - Enterprise...
Why Kubernetes? Cloud Native and Developer Experience at Zalando - Enterprise...
Henning Jacobs•1.8K views
Virtual Kubernetes Clusters on Amazon EKS by Jim Bugwadia
Virtual Kubernetes Clusters on Amazon EKSVirtual Kubernetes Clusters on Amazon EKS
Virtual Kubernetes Clusters on Amazon EKS
Jim Bugwadia•606 views
給 RD 的 Kubernetes 初體驗 by William Yeh
給 RD 的 Kubernetes 初體驗給 RD 的 Kubernetes 初體驗
給 RD 的 Kubernetes 初體驗
William Yeh•1.3K views

Recently uploaded

Introduction to Gradle by
Introduction to GradleIntroduction to Gradle
Introduction to GradleJohn Valentino
6 views•7 slides
Benefits in Software Development by
Benefits in Software DevelopmentBenefits in Software Development
Benefits in Software DevelopmentJohn Valentino
5 views•15 slides
Ports-and-Adapters Architecture for Embedded HMI by
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMIBurkhard Stubert
33 views•19 slides
ADDO_2022_CICID_Tom_Halpin.pdf by
ADDO_2022_CICID_Tom_Halpin.pdfADDO_2022_CICID_Tom_Halpin.pdf
ADDO_2022_CICID_Tom_Halpin.pdfTomHalpin9
5 views•33 slides
How to build dyanmic dashboards and ensure they always work by
How to build dyanmic dashboards and ensure they always workHow to build dyanmic dashboards and ensure they always work
How to build dyanmic dashboards and ensure they always workWiiisdom
14 views•13 slides
Playwright Retries by
Playwright RetriesPlaywright Retries
Playwright Retriesartembondar5
6 views•1 slide

Recently uploaded(20)

Introduction to Gradle by John Valentino
Introduction to GradleIntroduction to Gradle
Introduction to Gradle
John Valentino•6 views
Benefits in Software Development by John Valentino
Benefits in Software DevelopmentBenefits in Software Development
Benefits in Software Development
John Valentino•5 views
Ports-and-Adapters Architecture for Embedded HMI by Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert•33 views
ADDO_2022_CICID_Tom_Halpin.pdf by TomHalpin9
ADDO_2022_CICID_Tom_Halpin.pdfADDO_2022_CICID_Tom_Halpin.pdf
ADDO_2022_CICID_Tom_Halpin.pdf
TomHalpin9•5 views
How to build dyanmic dashboards and ensure they always work by Wiiisdom
How to build dyanmic dashboards and ensure they always workHow to build dyanmic dashboards and ensure they always work
How to build dyanmic dashboards and ensure they always work
Wiiisdom•14 views
Playwright Retries by artembondar5
Playwright RetriesPlaywright Retries
Playwright Retries
artembondar5•6 views
FOSSLight Community Day 2023-11-30 by Shane Coughlan
FOSSLight Community Day 2023-11-30FOSSLight Community Day 2023-11-30
FOSSLight Community Day 2023-11-30
Shane Coughlan•7 views
nintendo_64.pptx by paiga02016
nintendo_64.pptxnintendo_64.pptx
nintendo_64.pptx
paiga02016•6 views
Automated Testing of Microsoft Power BI Reports by RTTS
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
RTTS•10 views
Quality Engineer: A Day in the Life by John Valentino
Quality Engineer: A Day in the LifeQuality Engineer: A Day in the Life
Quality Engineer: A Day in the Life
John Valentino•7 views
aATP - New Correlation Confirmation Feature.pptx by EsatEsenek1
aATP - New Correlation Confirmation Feature.pptxaATP - New Correlation Confirmation Feature.pptx
aATP - New Correlation Confirmation Feature.pptx
EsatEsenek1•205 views
Programming Field by thehardtechnology
Programming FieldProgramming Field
Programming Field
thehardtechnology•6 views
Top-5-production-devconMunich-2023.pptx by Tier1 app
Top-5-production-devconMunich-2023.pptxTop-5-production-devconMunich-2023.pptx
Top-5-production-devconMunich-2023.pptx
Tier1 app•9 views
DRYiCE™ iAutomate: AI-enhanced Intelligent Runbook Automation by HCLSoftware
DRYiCE™ iAutomate: AI-enhanced Intelligent Runbook AutomationDRYiCE™ iAutomate: AI-enhanced Intelligent Runbook Automation
DRYiCE™ iAutomate: AI-enhanced Intelligent Runbook Automation
HCLSoftware•6 views
Generic or specific? Making sensible software design decisions by Bert Jan Schrijver
Generic or specific? Making sensible software design decisionsGeneric or specific? Making sensible software design decisions
Generic or specific? Making sensible software design decisions
Bert Jan Schrijver•7 views
Quality Assurance by interworksoftware2
Quality Assurance Quality Assurance
Quality Assurance
interworksoftware2•5 views
Airline Booking Software by SharmiMehta
Airline Booking SoftwareAirline Booking Software
Airline Booking Software
SharmiMehta•9 views
Transport Management System - Shipment & Container Tracking by Freightoscope
Transport Management System - Shipment & Container TrackingTransport Management System - Shipment & Container Tracking
Transport Management System - Shipment & Container Tracking
Freightoscope •5 views
.NET Deserialization Attacks by Dharmalingam Ganesan
.NET Deserialization Attacks.NET Deserialization Attacks
.NET Deserialization Attacks
Dharmalingam Ganesan•5 views
Electronic AWB - Electronic Air Waybill by Freightoscope
Electronic AWB - Electronic Air Waybill Electronic AWB - Electronic Air Waybill
Electronic AWB - Electronic Air Waybill
Freightoscope •5 views

High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

  • 1. Professorship for Computer Science Communication Services, Telecommunication Systems and Computer Networks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster HighLoad++ Conf, Moskow | Docker Meetup, Bamberg November 8, 2016 | November 17, 2016 @MathiasRenner | Hypriot.com
  • 2. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Content 1 Motivation 2 Why Kubernetes and Docker? 3 Related Work 4 Test Setup 5 Testing Scenarios 6 Results 7 Live Demo 8 Outlook 9 Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 2/30
  • 3. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 4. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 5. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 6. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 7. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 8. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 9. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 10. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 11. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular Own contribution for Docker on ARM High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  • 12. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 13. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 14. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 15. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 16. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 17. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 18. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 19. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 20. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 21. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 22. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 23. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch OpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  • 24. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 25. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 26. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 27. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 28. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 29. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 30. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 31. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. This work’s contribution: Extend tests of Nissen/Jensen and compare K8s against Docker Swarm. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  • 32. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Hardware Setup with PicoCluster Figure 1: Pico Cluster 5xRPi 3, https://www.picocluster.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 6/30
  • 33. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  • 34. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  • 35. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... Software Which OS? Raspbian vs. HypriotOS vs. ArchLinux Flashing (I stopped counting), device-init... Network: 2 interfaces per device, check DHCPs ... Update Software Monitoring system that supports all required capabilities Configure Desktop machine: No network interference, appropriate software to manage the cluster (terminator) Setup NTP SSH listen on two interfaces Setup Swapspace Automate, automate, automate! ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  • 36. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Docker Easy since included in HypriotOS. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 8/30
  • 37. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  • 38. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  • 39. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  • 40. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install Recently, setup became as easy as with Docker. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  • 41. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software stack for experiments Kubernetes Stack Docker Swarm Stack rpicloud/greeting-rpi rpicloud/greeting-rpi K8s@ARM Docker Docker HypriotOS HypriotOS High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 10/30
  • 42. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  • 43. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  • 44. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test Resilience Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  • 45. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resource Usage Table 1: First results of resource consumption measurements Sc. CPU usage, % Mem usage, % Proc CPU D., % Proc mem D., MB 1 0.44 4.21 0.05 2.47 2 1.43 / 0.83 7.87 / 7.4 1.1 / 0.44 57.45 / 2.42 3 14.74 / 2.01 34.21 / 12.68 5.76 / 1.31 479.51 / 160.77 Legend: Scenario 1: Fresh install, monit and Docker daemon running Scenario 2: After docker init Scenario 3: After starting K8s Also, in Scenario 3, Docker created 41 process children. In Scenario 1 & 2, Docker has 1. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 12/30
  • 46. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned And then the Big Bang... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 13/30
  • 47. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 48. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 49. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 50. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 51. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 52. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 → Both tools were basically not working to the minimum extent. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  • 53. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 2: Timeout of curl command against the cluster High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 15/30
  • 54. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 3: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 16/30
  • 55. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 4: Incoming network traffic Figure 5: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 17/30
  • 56. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 6: One of the resolved routing issues for Docker 1.12.2 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 18/30
  • 57. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Plan B... Throw most of the effort away and figure out a Plan B... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 19/30
  • 58. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Final Software stack of experiments Kubernetes Stack Docker Swarm Stack (Nissen/Jensen) rpicloud/greeting-rpi rpicloud/greeting-rpi K8s v1.2 + K8s@ARM v0.7 Docker v1.11.x Docker v1.12.3 ArchLinux ARM latest HypriotOS v1.1.0 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 20/30
  • 59. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 7: Breakpoints of load test with Kubernetes Test has performed with 10.000 req at eatch step. Stepsize=100 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 21/30
  • 60. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 8: Breakpoints of load test with Docker Swarm Mode Note that the blue and green line overlap very much. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 22/30
  • 61. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Load Test Table 2: Breakpoints of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 ca. 900 reqests/seconds ca. 600 reqests/seconds 5 ca. 1400 reqests/seconds ca. 900 reqests/seconds 10 ca. 1500 reqests/seconds ca. 900 reqests/seconds High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 23/30
  • 62. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 9: Successful requests onto the Kubernetes cluster with 1 replica Kubernetes: Total outage of 47 seconds Docker Swarm Mode: Total outage of 72 seconds (+53%) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 24/30
  • 63. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 10: Successful requests onto the cluster with 2 replica High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 25/30
  • 64. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resilience Test Table 3: Success rates of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 74.65% 61.74% 2 99.87% 99.85% 5 99.98% 99.93% High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 26/30
  • 65. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Live Demo: ”Let’s crash the cluster!” High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 27/30
  • 66. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
  • 67. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level Use single board computers with higher network I/O High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
  • 68. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  • 69. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  • 70. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  • 71. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  • 72. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Questions ? @MathiasRenner www.Hypriot.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 30/30