Data High Availability With TIDB
•
0 likes•40 views
This document discusses data high availability with TiDB. It provides an overview of TiDB's architecture including TiKV for data storage using Raft consensus, Placement Driver (PD) for orchestration, and TiFlash for analytics. It describes how TiDB uses labels to place regions across nodes to achieve high availability and fault tolerance. It also discusses election processes, replication, and automatic failover to maintain high availability of the TiDB cluster.
Recommended
Recommended
More Related Content
Similar to Data High Availability With TIDB
Similar to Data High Availability With TIDB (20)
More from Mydbops
More from Mydbops (20)
Recently uploaded
Recently uploaded (20)
Data High Availability With TIDB
- 1. Data High Availability With TiDB Oct 28th, 2023 Mydbops MyWebinar - 28 Kabilesh PR Co-Founder, Mydbops
- 2. Interested in Open Source technologies Pingcap Certified TiDB Associate Tech Speaker/Blogger AWS certified Database speciality AWS community builder / SME Co-Founder, Mydbops Kabilesh PR About Me
- 3. Mydbops at a glance 300+ Clients 3000+ Servers ISO & PCI DSS Certified 7 Yrs of Expertise AWS Advanced Consulting Partner TiDB PingCap Partner
- 4. Mydbops Services Consulting Services Managed Services MySQL, MongoDB, PostgreSQL, TiDB & Cassandra Targeted Engagement 24 x 7 DBA Team
- 5. Agenda What is TiDB ? Architecture Labels High Availability
- 6. What is TiDB ? An Open Source distributed SQL database by PingCAP • Scalable in both ways • Supports both Transactional and Analytical (HTAP) • Compatible with MySQL protocol • Highly available • Fault Tolerant •
- 7. Architecture
- 8. Architecture TiDB cluster - SQL layer TiKV - Storage layer Placement Driver(PD) - Orchestrator TiFlash - Analytical columnar storage
- 9. TiKV - Data Storage Stateful • Persistent storage with RocksDB • Distributed storage & transaction support • Strong Consistency • Min 3 nodes (HA by default) •
- 10. What is RAFT ? RAFT is a Consensus based method to replicate data and maintain HA • Reliable, Replicated , Redundant, And Fault-Tolerant (RAFT) • A Raft group has a leader and followers • Leader handles RW ops • Followers Participate in Election & Data Reads •
- 11. TiKV - Regions and Raft Region - Smallest data unit for replication • Each region is replicated 3 times and distributed among TiKV • Regions are automatically balanced across TiKV nodes • The replicas for one region forms a RAFT group • Default Region size is 96M •
- 12. Services (LB) Stateful Sets Operator Maturity TiKV - Regions and Raft
- 13. Replication With Regions Leader replicates log info to the followers • Wait for quorum ack and marks write as durable • From the logs data is synced to RocksDB KV • TiFlash sync from the log using Raft-learner •
- 14. Replication Process PROPOSE - Write request at leader written to Raftlog • APPEND - Raft-log is written locally to RAFTEngine • REPLICATE - Raft-log replicates to the followers • APPEND - Raft-log is written locally to RAFTEngine • COMMIT - ACK back to the leader on Durable Raft-log • APPLY - Write from RAFT log to TiKV RocksDB •
- 16. Election Scenarios RAFT always ensures to have only one leader for the regions Heart Beat Failures - N/W failures , node crash • New Region Split • Node Restart • Election Timeout •
- 17. Election Process Raft Maintains Terms, A leader will last till this Period Leader Emits Heart-beat to its followers When Heart Beat fails due to node-crash or N/W Followers change role to CANDIDATE also Increment the TERM Vote itself & then sends VOTING request to other nodes along the TERM value On MAJORITY it becomes the leader and sends back the HB
- 18. Election Process
- 19. Placement Driver - HA PD consists at least 3 NODES • Auto Failover with ETCD cluster • RAFT with ETCD to have strong consistency • PD Leader servers the requests • PD maintains the Labels •
- 21. Labels for High Availablity Its a way to tell TiDB cluster how to place regions • By default PD places the region randomly in TiKV nodes • With Labels we can guide PD to place regions based on DC, RACK and Nodes •
- 23. Use-Cases https://www.pingcap.com/customers/ SAAS Application IOT data , Time series Ecommerce Logistics Gaming Fintech
- 24. Limitations Stored procedures & Functions Triggers Events User-defined functions Fulltext and Spatial index Column-level privileges
- 25. Reach Us : Info@mydbops.com Thank You