During this joint webinar our speakers, Alkin Tezuysal, Technical Expert at Percona and Sugu Sougoumarane, Co-Founder and CTO at PlanetScale, will provide attendees with hands-on experience using Vitess using Kubernetes. They will start by providing a quick overview of Vitess – including key concepts and terminology, and then describe the deployment options for both in Kubernetes. The hands-on exercises will cover many of the key workflows with running a sharded system: – Moving from an unsharded system to a vertical split one (moving some tables to a different host) – Sharding large tables across several instances – Backups, Recoveries, and failovers (both planned and unplanned). Attendees are not expected to have any prior experience with Vitess but will find it useful if they have prior experience completing these tasks in a MySQL environment and general know-how on Kubernetes operations. The tutorial will be in three parts as follows: Introduction to Vitess History and background Setting up Vitess in Kubernetes Initial deployment and configurations Management Workflow Scalability Operational tasks with Vitess Vertical Split Horizontal Split Shard Management Performance Considerations Schema Changes

1. Introduction to Vitess
on Kubernetes for
MySQL
Alkin Tezuysal - Sr. Technical Manager, Percona
Sugu Sougoumarane - CTO/Co-Founder, PlanetScale
2020 Webinar Series - May 11th 2020

2. © 2020 Percona
Who are we?
Sr. Technical Manager at Percona
Open Source Database Evangelist
Global Database Operations Expert
Inspiring Technical and Strategic
Leader
Creative Team Builder
Speaker, Mentor, and Coach
Co-creator of Vitess, CTO at
PlanetScale.
Scalability at YouTube, PayPal in
the early days.
Argues about distributed systems
and consensus algorithms.
Occasionally blogs at
ssougou.blogspot.com.
2
Alkin Tezuysal - @ask_dba
Born to Sail, Forced
to Work
Sugu Sougoumarane - @ssougou
Storage Geek
@@ask_dba, @ssougou

3. © 2020 Percona
A Database
Clustering
System for
Horizontal
Scaling of
MySQL
3
@@ask_dba, @ssougou

4. © 2020 Percona
Introduction to
Vitess
4
@@ask_dba, @ssougou

5. © 2020 Percona
Vitess in CNCF
(Cloud Native
Computing
Foundation)
5
@@ask_dba, @ssougou

6. © 2020 Percona
True
Multi-Cloud
Clusters for
PlanetScaleDB
6
@@ask_dba, @ssougou

7. © 2020 Percona
Kubernetes on
Google Cloud
(GKE)
● Google Cloud's load-balancing for Compute Engine instances
● Node pools to designate subsets of nodes within a cluster for
additional flexibility
● Automatic scaling of your cluster's node instance count
● Automatic upgrades for your cluster's node software
● Node auto-repair to maintain node health and availability
● Logging and monitoring with Google Cloud's operations suite for
visibility into your cluster
7
@@ask_dba, @ssougou

8. © 2020 Percona
Installation and
Configuration
● Vitess on Google Cloud Platform Prerequisites
○ Google Kubernetes Engine (GKE)
○ Cluster with 7 nodes
○ K8s Version 1.15.11-gke.1
○ Helm Version 3
● Requirements
○ RBR
○ GTID
○ MySQL 5.6+ MariaDB 10.x+
8
@@ask_dba, @ssougou

9. © 2020 Percona
Create Vitess
Cluster GKE
9
@@ask_dba, @ssougou

10. © 2020 Percona
Setup Basics
10
@@ask_dba, @ssougou

11. © 2020 Percona
Configure
Cluster
11
@@ask_dba, @ssougou

12. © 2020 Percona
Connect to K8s
Cluster
12
@@ask_dba, @ssougou

13. © 2020 Percona
Download and
install Helm 3.X
$ wget https://get.helm.sh/helm-v3.1.2-linux-amd64.tar.gz
$ tar xvfz helm-v3.1.2-linux-amd64.tar.gz
$ sudo cp linux-amd64/helm /usr/local/bin/helm
$ go get vitess.io/vitess/go/cmd/vtctlclient
$ git clone git@github.com:vitessio/vitess.git
$ cd vitess/examples/helm
13
@@ask_dba, @ssougou

14. © 2020 Percona
Install Vitess
$ helm install vitess ../../helm/vitess -f 101_initial_cluster.yaml
NAME: vitess
LAST DEPLOYED: Thu May 7 13:55:49 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Release name: vitess
To access administrative web pages, start a proxy with:
kubectl proxy --port=8001
Then use the following URLs:
vtctld:
http://localhost:8001/api/v1/namespaces/default/services/vtctld:web/proxy/app/
vtgate:
http://localhost:8001/api/v1/namespaces/default/services/vtgate-zone1:web/proxy/
14
@@ask_dba, @ssougou

15. © 2020 Percona
Run Proxy and
access to Web
Portal(s)
To access administrative web pages, start a proxy with:
kubectl proxy --port=8001
Then use the following URLs:
vtctld:
http://localhost:8001/api/v1/namespaces/default/services/vtctld:web/proxy/app/
vtgate:
http://localhost:8001/api/v1/namespaces/default/services/vtgate-zone1:web/proxy/
15
@@ask_dba, @ssougou

16. © 2020 Percona
Vitess Control
Panel
16
@@ask_dba, @ssougou

17. © 2020 Percona
vtgate status
dashboard
17
@@ask_dba, @ssougou

18. © 2020 Percona
kubectl status
$ kubectl get pods,jobs
NAME READY STATUS RESTARTS AGE
pod/commerce-apply-schema-initial-z27sm 0/1 Completed 0 14m
pod/commerce-apply-vschema-initial-292nc 0/1 Completed 0 14m
pod/vtctld-c56d6c788-t9xlc 1/1 Running 0 14m
pod/vtgate-zone1-7d4957f6-49xsr 1/1 Running 0 14m
pod/zone1-commerce-0-init-shard-master-69fbc 0/1 Completed 0 14m
pod/zone1-commerce-0-replica-0 6/6 Running 0 14m
pod/zone1-commerce-0-replica-1 6/6 Running 0 14m
pod/zone1-commerce-0-replica-2 6/6 Running 0 14m
NAME COMPLETIONS DURATION AGE
job.batch/commerce-apply-schema-initial 1/1 59s 14m
job.batch/commerce-apply-vschema-initial 1/1 62s 14m
job.batch/zone1-commerce-0-init-shard-master 1/1 57s 14m
18
@@ask_dba, @ssougou

19. © 2020 Percona
Port-forwarding
kubectl port-forward service/vtctld 15000 15999 &
process_id1=$!
kubectl port-forward service/vtgate-zone1 15306:3306 15001 &
process_id2=$!
sleep 2
echo "You may point your browser to http://localhost:15000 for vtctld."
echo "You may point your browser to http://localhost:15001 for vtgate, use the
following aliases as shortcuts:"
echo 'alias vtctlclient="vtctlclient -server=localhost:15999"'
echo 'alias mysql="mysql -h 127.0.0.1 -P 15306"'
echo "Hit Ctrl-C to stop the port forwards"
wait $process_id1
wait $process_id2
19
@@ask_dba, @ssougou

20. © 2020 Percona
Connect to
MySQL
$ mysql
Handling connection for 15306
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 7
Server version: 5.7.9-Vitess Percona Server (GPL), Release 29, Revision 11ad961
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
mysql> show databases;
+-----------+
| Databases |
+-----------+
| commerce |
+-----------+
1 row in set (0.11 sec)
20
@@ask_dba, @ssougou

21. © 2020 Percona
Creating a
Keyspace
mysql> show tables;
+--------------------+
| Tables_in_commerce |
+--------------------+
| corder |
| customer |
| product |
+--------------------+
3 rows in set (0.11 sec)
$ mysql < ./vitess/examples/common/insert_commerce_data.sql
21
@@ask_dba, @ssougou

22. © 2020 Percona
Verify data
mysql> select * from corder; select * from customer; select * from product;
+----------+-------------+----------+-------+
| order_id | customer_id | sku | price |
+----------+-------------+----------+-------+
| 1 | 1 | SKU-1001 | 100 |
| 2 | 2 | SKU-1002 | 30 |
| 3 | 3 | SKU-1002 | 30 |
| 4 | 4 | SKU-1002 | 30 |
| 5 | 5 | SKU-1002 | 30 |
+----------+-------------+----------+-------+
5 rows in set (0.11 sec)
22
@@ask_dba, @ssougou

23. © 2020 Percona
Verify data
+-------------+--------------------+
| customer_id | email |
+-------------+--------------------+
| 1 | alice@domain.com |
| 2 | bob@domain.com |
| 3 | charlie@domain.com |
| 4 | dan@domain.com |
| 5 | eve@domain.com |
+-------------+--------------------+
5 rows in set (0.11 sec)
+----------+-------------+-------+
| sku | description | price |
+----------+-------------+-------+
| SKU-1001 | Monitor | 100 |
| SKU-1002 | Keyboard | 30 |
+----------+-------------+-------+
2 rows in set (0.11 sec)
23
@@ask_dba, @ssougou

24. © 2020 Percona
Similarities and
Differences
Similarities
● If Vitess was treated as MySQL Server, a Keyspace would be a
database.
● A Keyspace can have one or many shards
● Each Shard is one MySQL cluster: master+replicas
Differences
● If a table name is unique across all keyspaces, it can be accessed
without the qualifying it with db name
● You can “use `keyspace@master`” or “use `keyspace@replica`”
● Bypass shard routing with “use `keyspace:-80@master`”
24
@@ask_dba, @ssougou

25. © 2020 Percona
Moving Tables
Use case
Customer load is increasing, MySQL is reaching its limits. We want
to move the customer and corder tables to a separate MySQL
cluster, aka “unsharded keyspace”.
25
@@ask_dba, @ssougou

26. © 2020 Percona
Create customer
keyspace
keyspaces:
- name: "commerce"
shards:
- name: "0"
tablets:
- type: "replica"
vttablet:
replicas: 3
- name: "customer"
shards:
- name: "0"
tablets:
- type: "replica"
vttablet:
replicas: 3
26
@@ask_dba, @ssougou

27. © 2020 Percona
Bring up
customer
keyspace
$ helm upgrade vitess ../../helm/vitess/ -f 201_customer_tablets.yaml
Release "vitess" has been upgraded. Happy Helming!
NAME: vitess
LAST DEPLOYED: Sun May 10 14:21:14 2020
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
Release name: vitess
To access administrative web pages, start a proxy with:
kubectl proxy --port=8001
Then use the following URLs:
vtctld:
http://localhost:8001/api/v1/namespaces/default/services/vtctld:web/proxy/app/
vtgate:
http://localhost:8001/api/v1/namespaces/default/services/vtgate-zone1:web/proxy/
a
27
@@ask_dba, @ssougou

28. © 2020 Percona
Verify upgrade
zone1-customer-0-replica-0 2/2 Running 0 108s
zone1-customer-0-replica-1 2/2 Running 0 108s
zone1-customer-0-replica-2 2/2 Running 0 108s
mysql> show databases;
+-----------+
| Databases |
+-----------+
| commerce |
| customer |
+-----------+
2 rows in set (0.04 sec)
28
@@ask_dba, @ssougou

29. © 2020 Percona
Initiate move
tables
$ vtctlclient MoveTables -workflow=commerce2customer commerce customer
'{"customer":{}, "corder":{}}'
29
@@ask_dba, @ssougou

30. © 2020 Percona
Sanity check
mysql> use commerce:0
mysql> select count(*) from corder;
+----------+
| count(*) |
+----------+
| 5 |
+----------+
mysql> use customer:0
mysql> select count(*) from corder;
+----------+
| count(*) |
+----------+
| 5 |
+----------+
30
@@ask_dba, @ssougou

31. © 2020 Percona
VDiff
~/...vitess/examples/helm> vtctlclient VDiff customer.commerce2customer
Summary for corder: {ProcessedRows:5 MatchingRows:5 MismatchedRows:0
ExtraRowsSource:0 ExtraRowsTarget:0}
Summary for customer: {ProcessedRows:5 MatchingRows:5 MismatchedRows:0
ExtraRowsSource:0 ExtraRowsTarget:0}
31
@@ask_dba, @ssougou

32. © 2020 Percona
Switch Traffic
$ vtctlclient SwitchReads -tablet_type=rdonly customer.commerce2customer
$ vtctlclient SwitchReads -tablet_type=replica customer.commerce2customer
$ vtctlclient SwitchWrites customer.commerce2customer
32
@@ask_dba, @ssougou

33. © 2020 Percona
Clean-up $ vtctlclient DropSources customer.commerce2customer
33
@@ask_dba, @ssougou

34. © 2020 Percona
Reshard
Use case
Traffic has continued to grow, but the corder table is getting too big
to fit in a single database.
There is a relationship between corder and customer. It will be
beneficial to co-locate every orders with their customer rows.
Auto-inc won’t work. Solution: Vitess Sequences.
34
@@ask_dba, @ssougou

35. © 2020 Percona
Creating
sequences
301_customer_sharded.yaml (commerce schema)
create table customer_seq(id int, next_id bigint, cache bigint, primary key(id))
comment 'vitess_sequence';
insert into customer_seq(id, next_id, cache) values(0, 1000, 100);
create table order_seq(id int, next_id bigint, cache bigint, primary key(id)) comment
'vitess_sequence';
insert into order_seq(id, next_id, cache) values(0, 1000, 100);
35
@@ask_dba, @ssougou

36. © 2020 Percona
Creating
sequences
301_customer_sharded.yaml (customer VSchema)
{
"tables": {
"customer_seq": {
"type": "sequence"
},
"order_seq": {
"type": "sequence"
},
"product": {}
}
}
36
@@ask_dba, @ssougou

37. © 2020 Percona
301_customer_sharded.yaml (commerce
VSchema)
{
"sharded": true,
"vindexes": {
"hash": {
"type": "hash"
}
},
"tables": {
"customer": {
"column_vindexes": [
{
"column": "customer_id",
...
"corder": {
"column_vindexes": [
{
"column": "customer_id",
"name": "hash"
}
],
"auto_increment": {
"column": "order_id",
"sequence": "order_seq"
}
}
}
}
Primary
Vindexes, aka
Sharding Keys
37
@@ask_dba, @ssougou

38. © 2020 Percona
Apply Changes $ helm upgrade vitess ../../helm/vitess/ -f 301_customer_sharded.yaml
38
@@ask_dba, @ssougou

39. © 2020 Percona
- name: "customer"
shards:
- name: "0"
tablets:
- type: "replica"
vttablet:
replicas: 3
- name: "-80"
tablets:
- type: "replica"
vttablet:
replicas: 3
copySchema:
source: "customer/0"
- name: "80-"
tablets:
- type: "replica"
vttablet:
replicas: 3
copySchema:
source: "customer/0"
Create Target
Shards
39
@@ask_dba, @ssougou

40. © 2020 Percona
Apply Changes $ helm upgrade vitess ../../helm/vitess/ -f 302_new_shards.yaml
40
@@ask_dba, @ssougou

41. © 2020 Percona
Verify cluster
zone1-customer-80-x-replica-0 2/2 Running 0 115s
zone1-customer-80-x-replica-1 2/2 Running 0 115s
zone1-customer-80-x-replica-2 2/2 Running 0 115s
zone1-customer-x-80-init-shard-master-6zhhn 0/1 Completed 0 114s
zone1-customer-x-80-replica-0 2/2 Running 0 114s
zone1-customer-x-80-replica-1 2/2 Running 0 114s
zone1-customer-x-80-replica-2 2/2 Running 0 114s
41
@@ask_dba, @ssougou

42. © 2020 Percona
Initiate
Resharding
$ vtctlclient Reshard customer.cust2cust '0' '-80,80-'
42
@@ask_dba, @ssougou

43. © 2020 Percona
mysql> use customer:0
mysql> select count(*) from corder;
+----------+
| count(*) |
+----------+
| 5 |
+----------+
mysql> use customer:-80
mysql> select count(*) from corder;
+----------+
| count(*) |
+----------+
| 4 |
+----------+
mysql> use customer:80-
mysql> select count(*) from corder;
+----------+
| count(*) |
+----------+
| 1 |
+----------+
Sanity Check
43
@@ask_dba, @ssougou

44. © 2020 Percona
VDiff
~/...vitess/examples/helm> vtctlclient VDiff customer.cust2cust
Summary for corder: {ProcessedRows:5 MatchingRows:5 MismatchedRows:0
ExtraRowsSource:0 ExtraRowsTarget:0}
Summary for customer: {ProcessedRows:5 MatchingRows:5 MismatchedRows:0
ExtraRowsSource:0 ExtraRowsTarget:0}
44
@@ask_dba, @ssougou

45. © 2020 Percona
Switch over
$ vtctlclient SwitchReads -tablet_type=rdonly customer.cust2cust
$ vtctlclient SwitchReads -tablet_type=replica customer.cust2cust
$ vtctlclient SwitchWrites customer.cust2cust
45
@@ask_dba, @ssougou

46. © 2020 Percona
Down Shard 0
$ helm upgrade vitess ../../helm/vitess/ -f 306_down_shard_0.yaml
$ vtctlclient DeleteShard -recursive customer/0
46
@@ask_dba, @ssougou

47. © 2020 Percona
Tear-down
Cluster
$ helm delete vitess
$ kubectl delete pvc -l "app=vitess"
$ kubectl delete vitesstoponodes --all
47
@@ask_dba, @ssougou

48. © 2020 Percona
Q&A
48
@@ask_dba, @ssougou

49. @ 2020 Percona
Introduction to Vitess on Kubernetes for MySQL - Part I of III
Vitess.io
PlanetScale
Operating Vitess – Square
Cloud Native Computing Foundation: Home Page
CNCF Cloud Native Interactive Landscape
PlanetScaleDB delivers true multi-cloud, multi-region clusters on AWS, GCP, and Azure - Blog
Vitess Demo
References
49