Raghavendra Prabhu presented on orchestrating Percona XtraDB Cluster (PXC) with Kubernetes. Some key points:
- Kubernetes provides horizontal scaling, self-healing, automated rollouts/rollbacks, service discovery, storage orchestration and more.
- In Kubernetes, PXC nodes would be deployed as pods with a replication controller to maintain a set number of pods. Services provide load balancing to the pods.
- Demonstrated deploying a basic PXC cluster on Kubernetes, including creating a network, cluster, service, replicating pods from a template, and exposing ports.
- Challenges include load balancing for state transfers between nodes and ensuring nodes are
9. Rationale
➔ Nodes v/s Hosts
◆ Resource-based
➔ Reusable components
◆ Monitoring and Tracing
◆ DNS and Service Discovery
◆ Logging
◆ Metrics
◆ Scheduler
➔ Agnostic
➔ Roles
10. The Fit
➔ Layered
◆ Client - Server
◆ Multi-layered
➔ Scaling
◆ Horizontal and Vertical
● Preferred?
● Need for vertical
11. The Fit
➔ Statelessness
◆ Planes of logic:
● Control Plane
● Data Plane : Storage
◆ Anti-pattern for containers
➔ Elasticity
◆ Elastic Scalability
● Scaling down
12. Declarative vs Imperative
➔ Configurable mgmt
◆ Puppet, Nix, Terraform
➔ Microservices
◆ What runs on my laptop
● What runs on server
● Reproducibility
➔ 12-factor app
➔ Composability
➔ Immutable deployment artifact
13. Containers
● What is a container and why should I care
○ Operating system virtualization
● Isolation
○ Hierarchies of isolation - application, cgroups,
namespaces, seccomp…
● Unikernels and VMs
○ Role?
○ MirageOS, Rump kernel
● Some - LXC / LXD, Docker*, Rocket*, runc, jails, solaris
zones, lmcty, systemd-nspawn
14. Galera - really short intro!
➔ MySQL and WSREP api
➔ Galera plugin
➔ Group communication
➔ Synchronous replication
◆ ‘Virtually’
➔ EVS
➔ Certification-based
◆ Optimistic Concurrency
➔ Automatic Node Provisioning
15. Galera - really short intro!
➔ CAP theorem and Galera
◆ CP
➔ How does it fit
◆ Others
◆ Idempotency
➔ Stateless?
◆ Symmetric
◆ Replicas - Cassandra et.al.
◆ MySQL Cluster
➔ Maintenance of Quorum
16. Orchestration
● SOA def
○ ‘Stitching’
○ ‘Composing’
● Automation?
● Choreography
● Best of both worlds
17. Kubernetes
● Started as orchestrator
○ Is an ecosystem for containers
● Horizontal Scaling
● Self-healing
○ Chaos-monkey
● Latest issue
○ Rolling update in clusters
○ How K8s solves this
18. Kubernetes
● Bin packing
● Automated rollouts and rollbacks
● Secret management
○ Elegant
● Storage orchestration
● Service discovery and load balancing
○ Underrated
19. Kubernetes: API
● Consistent and Versioned
○ Very important glue
● Composable
● Developed with Swagger
● API Groups
● Supports both declarative and imperative
○ Rolling-update / Daemon Sets
31. Services
● Don’t commingle with `microservices`
○ Think of endpoints.
● Layering architecture
○ Logical address of subset of pods
● Communication
○ Environment
■ Ordering requirement
■ Discovery
○ DNS
■ Issues with DNS
32. Services: in general
● Potential issues
○ Staleness
○ Live HUP-ing
○ Propagation
● Haproxy
○ Reload configuration.
○ Solved at Yelp with linux qdiscs.
● Flux from Weave
35. Networking
● Docker-style linking
● Proxy for Pods
● Types
○ Pod to Pod
○ Pod to Service
○ Intra-Pod
○ External to Service
● Providers:
○ OpenVSwitch / Flannel / Calico / Weave / Google
38. Deployment
● Declare and build individual Galera/PXC nodes.
○ Keep it minimal and simple
○ No assumptions
● Without Kubernetes
○ Docker-compose
■ Possible issues
● Galera node ⇔ Pod
○ Haproxy
○ xinetd
39. Deployment
● Basic Steps:
○ Create a ‘flat’ network - 10.0.0.0/24
○ Create a ‘cluster’ - zone
○ Create a service endpoint.
■ Internal service - 3306/4567/4568.
■ External service - 3306/3306(?).
■ Expose the external.
■ Session affinities.
40. Deployment
● Next:
○ Bootstrap a node Pod from a template.
■ Query existing with selector.
○ Start rest of nodes from template.
■ Point to Service with selector.
■ Replication controller
○ Volumes
41. Deployment - Implications
● Load balancing in state transfers
● Respawning of nodes on timeout
○ May not be same nodes.
● kubectl to manage
○ Puppet etc. also have modules now.
● Separation of client and cluster traffic
42. Case Study: Safe restarts
● Highly available
● Unattended autonomous
○ and Imperative
● Restarts - services and nodes
● More of a orchestration than choreographing
● Randomness?
● Distributed locks
● Low impedance