Hybrid Cloud with Kubernetes Federation

container-solutions.com info@container-solutions.com Kubernetes Federation @ ContainerConf @michmueller_
info@container-solutions.com
container-solutions.com
Federation
Benefits &
Use Cases
16.11.2017
Michael Mueller
@michmueller_

British
Airways
May 27, 2017
Datacenter
shutdown
UPS Failure in
one datacenter

Microsoft
Azure
Nov 18, 2014
Storage outage
11 hours
11 DCs

Amazon Web
Services
Feb 28, 2017
S3 outage in
us-east-1
4 hours -
estimated cost:
$310 million
https://techcrunch.com/2017/02/28/amazon-aws-s3-outage-is-breaking-things-for-a-lot-of-websites-and-apps/

Why such wide-spread impact?
■ Most users had all-eggs-in-one-basket design
■ Applications running from a single location
■ Stand-by just not working

Hybrid Cloud!
“Everything fails, all the time”.1
■ Eliminate risks associated with business continuity of a single
provider/region/failure domain
■ Achieve the needed/required uptime with operating redundantly in
multiple failure domains
1
Werner Vogels, Amazon CTO

“Hybrid” means...
It’s hybrid if…
■ private cloud and public cloud services are combined or multiple clouds
■ it feels like one single environment
It’s not hybrid if…
■ a few developers at a company use some public cloud services
■ you use a SaaS application with no connection to the private cloud

Why Kubernetes Federation ?
■ Kubernetes is vendor independent (Project under the CNCF)
■ Can run anywhere
■ Underlying infrastructure can be changed at any point in time
■ Bridges the gap between on-prem and cloud
■ Single control plane
■ Can schedule workload automatically across DC/clouds/regions/…
■ Can deal with outages of DC/clouds/regions/…

Real world use cases

Typical enterprise
■ Some managed
datacenters
■ Interconnections
in between
■ Active / Passive
setup
http://mdcc.cs.berkeley.edu/mdcc_logo.pn
g

Failover
■ Semi automated
Failover
■ Route traffic from
one DC to
another

Started a
hybrid cloud
project
■ Opened an
AWS account
■ Some test
machines
running
■ Scale out is a
must, but not
implemented

Solution

Kubernetes Federation
■ Application
orchestrated
across individual
clusters
■ Simple
maintenance of
clusters
■ Single federated
management
App
On PremLondon (AWS)
St. Ghislain (GCE)
App App
Global VIP
Client

Application Failover
■ Application is
already
distributed
(Active/Active)
■ DC is marked as
down and traffic
is routed to the
others
■ Transparent
App
London (AWS) St. Ghislain (GCE)
App App
Global VIP
Client
St. Ghislain (GCE)

Obeying
regulatory
restrictions
“classic way”
■ Data needs to
be in e.g. UK
■ Traffic is routed
to the DC and
stays there

Obeying regulatory
restrictions
App
On PremLondon (AWS) St. Ghislain (GCE)
App App
Global VIP
Client
■ Data needs to
reside in e.g. UK
■ Clusters in UK get
labeled
■ App is only
scheduled on
clusters labeled
accordingly

Situation
■ Operating in both the People’s Republic of China and the rest of the world
(AWS Beijing and Singapore)
■ Deal with a combination of technologies and legislative actions
■ Mitigate the potential risk of outages caused by having data centres, and
customers, on both sides of the GFW
■ Adopted containers and microservices already
■ Involved running microservices in a somewhat manual fashion across
each AWS region leveraging various scripts and tools
■ Wanted a scheduler to manage their workloads
■ Challanges: Mostly specific to the GFW
■ E.g.: Default container registries (Google) are blocked

Old - Setup

New - Setup

Benefits
■ Run applications at global scale with a single federated view
■ Cope with failures from machine to cluster/region/provider/DC level
■ Schedule capacity based on location
■ Escape provider lock-in

How federation works

Requirements
■ Multiple Kubernetes Clusters
■ A Federation Control Plane
■ DNS Provider
■ Bonus points: A globally distributed load balancing solution

High-level architecture
Image courtesy of CoreOS

Global
Loadbalancing

■ Google’s Global Load Balancer
■ Ideal solution with multiple clusters on Google Cloud
■ No solution for “Hybrid Cloud” (yet)
■ Backplane.io
■ Developer focused, very very new...
■ Working Kubernetes and Docker Swarm integration
■ DNS based (e.g. AWS Route53)
■ Usual “DNS Problems”, e.g. misbehaving clients ignoring TTLs
■ Failover times dependent on DNS TTLs and intermediate caching
■ “Classic” Content Delivery Networks
■ Currently, manual configuration needed
■ Best option for Hybrid installations now
Options for global loadbalancing

Good practices
■ Consider eventual consistency in your architecture
■ Asynchronous writes via message queues (e.g. Event Sourcing)
■ or globally distributed databases like Google Cloud Spanner
■ Replicate blob stores over multiple regions
■ prevents problems like the S3 outage in February

Federation Demo

Caveats & Limitations
■ Still under heavy development
■ Not all functionality is currently available in Federation
■ Single federated control plane
■ Increased network bandwidth and cost
■ A bug in the federation control plane could potentially impact all clusters

Questions?

Stay in touch!
Twitter
@michmueller_
@containersoluti
Blog
container-solutions.com/blog
Mail
m.mueller@container-solutions.com

Hybrid Cloud with Kubernetes Federation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Hybrid Cloud with Kubernetes Federation

Similar to Hybrid Cloud with Kubernetes Federation (20)

Recently uploaded

Recently uploaded (20)

Hybrid Cloud with Kubernetes Federation