The document discusses the history and architecture of TiDB, a NewSQL database developed by PingCAP. It highlights the growing popularity of TiDB, with over 300 deployments and features like cloud-native support, state management through the operator pattern, and a custom resource definition for managing clusters. The presentation also outlines the uses of TiDB in various applications, emphasizing its scalability and strong consistency.

1. “Smooth” Operator
Kevin Xu (@kevinsxu; kevin@pingcap.com)

2. Agenda
● History and Community
● Technical Walkthrough
● Operator to Run Stateful App

3. A little about PingCAP
● Founded in April 2015 by 3
infrastructure engineers
● Created and maintains TiDB, TiKV
● Offices throughout North America
and China

4. PingCAP.com
Community
Stars
● TiDB: 17,000+
● TiKV: 4,500+
Contributors
● TiDB: 230+
● TiKV: 120+
TiDB DevCon:
● 700+

5. PingCAP.com
Who’s Using TiDB?
300+
Deployments

6. Use Cases
1. Approaching the maximum size for MySQL
on a single server. Debating whether or not
to shard.
2. Already sharded MySQL, but having a hard
time doing analytics on up-to-date data.

7. PingCAP.com
Mobike + TiDB
● 200 million users
● 200 cities
● 9 million smart bikes
● ~30 TB / day

8. Technical Walkthrough

9. Technical Inspiration
TiDB is a NewSQL database that speaks the MySQL protocol
It is not based on the MySQL source code
It is an ACID/strongly consistent database
The inspiration is Google Spanner + F1
It separates SQL processing and Storage into separate components
Both of them are independently scalable
The SQL processing layer is stateless
It is designed for both Transaction and Analytical Processing (HTAP)

10. How “Cloud Native” is TiDB?
Cloud native technologies empower organizations to build and run scalable applications in modern,
dynamic environments such as public, private, and hybrid clouds. Containers, service meshes,
microservices, immutable infrastructure, and declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and observable.
Combined with robust automation, they allow engineers to make high-impact changes frequently and
predictably with minimal toil.
The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and
sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art
patterns to make these innovations accessible for everyone.

11. TiDB
TiDB
Region 1 L
TiKV Node 1
Region 2
Region 3
Region 4
Region 2 L
TiKVNode 3
Region 3
Region 4 L
Region 1
Region 4
TiKV Node 2
Region 3 L
Region 2
Region 1
TiKV Cluster
PD Cluster

12. PingCAP.com
Cloud Native Ecosystem
● Prometheus
○ (maintains Rust implementation:
https://github.com/pingcap/rust-prometheus)
● gRPC
○ (maintains Rust implementation: https://github.com/pingcap/grpc-rs)
● etcd
○ we contributed the “Learner” feature

13. Row + Column Storage (Announced Jan 2019)
Spark Cluster
TiDB
TiDB
Region 1
TiKV Node 1
Region 2
Region 3
Region 4
Region 2
TiKV Node 3
Region 3
Region 4
Region 1
Region 4
TiKV Node 2
Region 3
Region 2
Region 1
TiFlash Node 2
TiFlash Extension Cluster TiKV Cluster
TiSpark
Worker
TiSpark
Worker
TiFlash Node 1

14. PingCAP.com
TiKV: The Storage Foundation
Region 5
Region 1
Region 3
TiKV node 1
Store 1
Region 4
gRPC
Region 1
Region 2
TiKV node 2
Store 2
Region 3
gRPC
Region 3
Region 1
Region 5
TiKV node 3
Store 3
gRPC
Region 5
Region 1
Region 2
TiKV node 4
Store 4
gRPC
Client
PD 1
PD 2
PD 3
Placement
Driver
Raft GroupRegion 4
Region 4

15. Operator

16. Operator History
● Operator pattern inspired by CoreOS...(now Redhat...(now
IBM))
● Introduced in 2016, Operator Framework in 2018
● TiDB Operator (2017) predated Operator
Framework
● Growing popularity in database community:
○ https://thenewstack.io/databases-operators-bring-stateful-workloads-to-kubernet
es/

17. PingCAP.com
TiDB Operator Architecture
API ServerController ManagerScheduler
Kubernetes Core
TiDB Controller Manager
TiDB Cluster Controller
PD Controller
TiKV
Controller
TiDB
Controller
TiDB Scheduler:
TiDB Cloud Manager
API Gateway
Control Plane
Cost Controller
Kube Scheduler
TiDB Scheduler

18. Features
● Manage multiple clusters
● Safe scaling
● Network/Local Persistent Volume (PV) support
● Automatic monitoring
● Rolling updates
● Automatic failover

19. How TiDB manages state -- StatefulSet
Unlike Deployment, StatefulSet...
● Guarantees ordering and uniqueness of pods
● Gives “sticky” identity -- network and storage (CNI, CSI)
● *No* interchangeable pods (always map the volume back to
the same pod)

21. How TiDB manages state -- CRD
● Custom Resource Definition (CRD):
○ An application-specific YAML file
○ User write its domain operation logic in CRD
○ Simple to implement and deploy
● (There is another way):
○ API Aggregation:
■ More control, more powerful but…
■ Hard to deploy, not well-supported by k8s engines

22. How TiDB manages state -- Custom Controller
Spec:
component:
image:
replicas:
...
Status:
image
replicas
state
CRD
(provided by user)
Controller
Cluster State

23. How TiDB manages state
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
name: demo
spec:
pd:
image: pingcap/pd:v2.1.0
replicas: 3
requests:
cpu: “4”
memory: “8Gi”
…
tikv:
image: pingcap/tikv:v2.1.0
…
status:
tikv:
stores:
“5”:
podName: demo-tikv-2
state: Up
...
type Manager interface {
Sync(*TidbCluster) error
}

24. PingCAP.com
All Open Sourced
https://github.com/pingcap/tidb-operator

25. PingCAP.com
On GCP Marketplace

26. PingCAP.com
TiDB, Managed As A Cloud Service
Early Access: https://www.pingcap.com/tidb-cloud/

27. Thank You
@kevinsxu; kevin@pingcap.com