Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Professorship for Computer Science
Communication Services, Telecommunication Systems and Computer Networks
High Availabili...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Motivation
Why Kubernetes and
Docker?
Related Work
Test Setup
Testing Scenarios
Results
Live Demo
Outlook
Lessons Learned
...
Upcoming SlideShare
Loading in …5
×

High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

1,523 views

Published on

The new Docker Swarm Mode claims to ship all features required to make a service in a cluster highly available. Thus, next to Kubernetes, Docker could be the second reasonable orchestration tool available that can be deployed on a Raspberry Pi cluster. But let's not trust the release notes – let's see how the reality looks like! Which of the two will win?

Published in: Software

High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster

  1. 1. Professorship for Computer Science Communication Services, Telecommunication Systems and Computer Networks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster HighLoad++ Conf, Moskow | Docker Meetup, Bamberg November 8, 2016 | November 17, 2016 @MathiasRenner | Hypriot.com
  2. 2. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Content 1 Motivation 2 Why Kubernetes and Docker? 3 Related Work 4 Test Setup 5 Testing Scenarios 6 Results 7 Live Demo 8 Outlook 9 Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 2/30
  3. 3. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  4. 4. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  5. 5. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  6. 6. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  7. 7. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  8. 8. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  9. 9. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  10. 10. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  11. 11. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Motivation Observing paradigm shift: From large to small devices, in size and power. Architecture size shrinks Energy efficiency increases Low-cost hardware availability increases → Internet of Things Contributing factors Educational added value (physical, bare metal). Same as big machines, just less powerful. Cluster for Distributed Systems/Microservices/SOA Very popular Own contribution for Docker on ARM High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 3/30
  12. 12. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  13. 13. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  14. 14. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  15. 15. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  16. 16. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  17. 17. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  18. 18. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  19. 19. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  20. 20. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  21. 21. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  22. 22. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  23. 23. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Why Kubernetes and Docker Docker Official support on ARM available since Aug 18, 2016 Lightweight Announced HA features that are very basic, but complete for enough use cases Kubernetes (K8s) Recently support for ARM has been extended Known as very reliable (based on Google’s Borg DC Manager) Not that ressource-intensive What about...? Mesos: Requires too much resources (Netflix got it working), complex setup Nomad: Still experimental, in general and on ARM arch OpenStack, Pacemaker, Marathon, Apache Spark, Open MPI, ROOT... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 4/30
  24. 24. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  25. 25. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  26. 26. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  27. 27. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  28. 28. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  29. 29. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  30. 30. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  31. 31. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Related Work In 2012: first public project about cloud computing with Raspberry Pis (the first model) has been started by researchers at the University of Glasgow LXC was state of the art, everything non-container virtualization too resource-hungry Largest project: Bolzano University with 300 RPis In 2015: Turning point: Lucas K¨aldstr¨om and University of Glasgow started at same time porting Kubernetes to ARM - independently. Lucas implemented to most features and recently became Kubernetes maintainer. In June 2016: Kaspar Nissen and Martin Jensen from Ahrhus University were first to run comprehensive load and high availability tests with Kubernetes on Raspberry Pis. Most versatile investigation about Cloud Computing at small scale. This work’s contribution: Extend tests of Nissen/Jensen and compare K8s against Docker Swarm. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 5/30
  32. 32. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Hardware Setup with PicoCluster Figure 1: Pico Cluster 5xRPi 3, https://www.picocluster.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 6/30
  33. 33. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  34. 34. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  35. 35. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Setup aspects Goal: Meet all requirements for good experiments. Hardware All SD cards of same type Check network switch Isolate network ... Software Which OS? Raspbian vs. HypriotOS vs. ArchLinux Flashing (I stopped counting), device-init... Network: 2 interfaces per device, check DHCPs ... Update Software Monitoring system that supports all required capabilities Configure Desktop machine: No network interference, appropriate software to manage the cluster (terminator) Setup NTP SSH listen on two interfaces Setup Swapspace Automate, automate, automate! ... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 7/30
  36. 36. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Docker Easy since included in HypriotOS. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 8/30
  37. 37. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  38. 38. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  39. 39. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  40. 40. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software Setup: Kubernetes Using project: ”Kubernetes-on-arm”by Lucas K¨aldstr¨om Bleeding edge: Setup non-deterministic and unreliable. Many trials, experiments and bug shooting. Installed Kubernetes via automated interactive call: /bin/echo -e " rpi-3nhypriotosnnode-greennEurope/Berlinn nnynY" | sudo kube-config install Recently, setup became as easy as with Docker. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 9/30
  41. 41. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Software stack for experiments Kubernetes Stack Docker Swarm Stack rpicloud/greeting-rpi rpicloud/greeting-rpi K8s@ARM Docker Docker HypriotOS HypriotOS High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 10/30
  42. 42. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  43. 43. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  44. 44. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Testing Scenarios Resource Usage Load Test Resilience Test High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 11/30
  45. 45. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resource Usage Table 1: First results of resource consumption measurements Sc. CPU usage, % Mem usage, % Proc CPU D., % Proc mem D., MB 1 0.44 4.21 0.05 2.47 2 1.43 / 0.83 7.87 / 7.4 1.1 / 0.44 57.45 / 2.42 3 14.74 / 2.01 34.21 / 12.68 5.76 / 1.31 479.51 / 160.77 Legend: Scenario 1: Fresh install, monit and Docker daemon running Scenario 2: After docker init Scenario 3: After starting K8s Also, in Scenario 3, Docker created 41 process children. In Scenario 1 & 2, Docker has 1. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 12/30
  46. 46. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned And then the Big Bang... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 13/30
  47. 47. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  48. 48. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  49. 49. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  50. 50. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  51. 51. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  52. 52. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Big Bang occurred. Kernel panics and systemd services/daemons crashed. Often reconfiguration and reboots required. Mesh routing: Produce timeouts in both K8s & Docker. Bugsnag: ”The intelligent routing was very spotty, sometimes hanging during a request, and sometimes not even accepting inbound connections at all.”1 → Both tools were basically not working to the minimum extent. 1http://blog.bugsnag.com/container-orchestration-with-docker-swarm-mode High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 14/30
  53. 53. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 2: Timeout of curl command against the cluster High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 15/30
  54. 54. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 3: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 16/30
  55. 55. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 4: Incoming network traffic Figure 5: Incoming network traffic High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 17/30
  56. 56. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Network Issues Figure 6: One of the resolved routing issues for Docker 1.12.2 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 18/30
  57. 57. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Plan B... Throw most of the effort away and figure out a Plan B... High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 19/30
  58. 58. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Final Software stack of experiments Kubernetes Stack Docker Swarm Stack (Nissen/Jensen) rpicloud/greeting-rpi rpicloud/greeting-rpi K8s v1.2 + K8s@ARM v0.7 Docker v1.11.x Docker v1.12.3 ArchLinux ARM latest HypriotOS v1.1.0 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 20/30
  59. 59. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 7: Breakpoints of load test with Kubernetes Test has performed with 10.000 req at eatch step. Stepsize=100 High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 21/30
  60. 60. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Load Test Figure 8: Breakpoints of load test with Docker Swarm Mode Note that the blue and green line overlap very much. High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 22/30
  61. 61. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Load Test Table 2: Breakpoints of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 ca. 900 reqests/seconds ca. 600 reqests/seconds 5 ca. 1400 reqests/seconds ca. 900 reqests/seconds 10 ca. 1500 reqests/seconds ca. 900 reqests/seconds High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 23/30
  62. 62. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 9: Successful requests onto the Kubernetes cluster with 1 replica Kubernetes: Total outage of 47 seconds Docker Swarm Mode: Total outage of 72 seconds (+53%) High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 24/30
  63. 63. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results - Resilience Test Figure 10: Successful requests onto the cluster with 2 replica High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 25/30
  64. 64. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Results – Resilience Test Table 3: Success rates of Kubernetes and Docker Swarm Mode Replica Count Kubernetes Docker Swarm Mode 1 74.65% 61.74% 2 99.87% 99.85% 5 99.98% 99.93% High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 26/30
  65. 65. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Live Demo: ”Let’s crash the cluster!” High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 27/30
  66. 66. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
  67. 67. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Outlook Add and test high available storage (GlusterFS, Ceph, HDFS) and extend applications to a microservices level Use single board computers with higher network I/O High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 28/30
  68. 68. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  69. 69. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  70. 70. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  71. 71. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Lessons Learned Don’t trust! Measure! Don’t trust! Measure! Don’t trust! Measure! High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 29/30
  72. 72. Motivation Why Kubernetes and Docker? Related Work Test Setup Testing Scenarios Results Live Demo Outlook Lessons Learned Questions ? @MathiasRenner www.Hypriot.com High Availability Performance of Kubernetes and Docker Swarm on a Raspberry Pi Cluster | @MathiasRenner | KTR 30/30

×