Download as PDF, PPTX
Uploaded byMarian Marinov
Thinking about highly-available systems and their setup
The document discusses the setup of highly-available systems, covering basics of distributed versus high availability (HA) systems, applications, networks, and storage solutions. It provides insights on optimal configurations for databases and session handling, emphasizing technologies like MySQL, Redis, and various file systems for data consistency. Additionally, it highlights decision-making tools such as HashiCorp Nomad and Kubernetes for managing these systems.
More Related Content
Similar to Thinking about highly-available systems and their setup
Thinking about highly-available systems and their setup
- 1. Setting up highly-available systems MarianMarinov <mm@yuhu.biz> Marian Marinov <mm@yuhu.biz>
- 2. Who am I? Whoam I? ➢ Sysadmin with more then 25y of experience ➢ Director of Engineering at Web Hosting Canada ➢ a FOSS dude :)
- 3. Agenda Agenda ➢ Basics ➢ Application ➢Network ➢ Storage ➢ Decision making
- 4. Basics Basics ➢ Distributed vs.HA systems
- 5. Basics - Distributed Basics- Distributed ➢ Distributed vs. HA systems DataCenter 1 DataCenter 2
- 6. Basics - Distributed Basics- Distributed ➢ Distributed vs. HA systems DataCenter 1 DataCenter 2
- 7. Basics - Distributed Basics- Distributed ➢ Distributed vs. HA systems DataCenter 1 DataCenter 2
- 8. Basics - HAsystems Basics - HA systems ➢ Distributed vs. HA systems
- 9. Basics - HAsystems Basics - HA systems ➢ Distributed vs. HA systems
- 10. Basics - HAsystems Basics - HA systems ➢ Distributed vs. HA systems ➢ Most software is not build for either of them
- 11. Application Application ➢ Static webapplication like WordPress ➢ Single DB ➢ Sessions ➢ Uploaded content
- 12. Application - DB Application- DB ➢ MySQL Replication works ➢ Combine it with: ➢ MariaDB MaxScale ➢ MySQL Router ➢ or ProxySQL
- 13. Application - Sessions Application- Sessions ➢ Redis/Valkey/DragonflyDB + HAproxy or Sentinel
- 14. Application - Sessions Application- Sessions ➢ Redis/Valkey/DragonflyDB + HAproxy or Sentinel ➢ do not use MySQL for sessions
- 15. Application - Sessions Application- Sessions ➢ Redis/Valkey/DragonflyDB + HAproxy or Sentinel ➢ do not use MySQL for sessions ➢ do not use a shared/distributed FS
- 16. Application - shareddata Application - shared data ➢ Eventual consistency ➢ Strong consistency
- 17. Application - shareddata Application - shared data ➢ Eventual consistency - rsync :) ➢ osync ➢ bsync ➢ unison ➢ unisync ➢ syncthing
- 18. Application - shareddata Application - shared data ➢ Strong consistency - file system ➢ DRBD+OCFS2 ➢ OCFS2/GFS2 with remote block storage ➢ OrangeFS (previously PVFS2) ➢ GlusterFS ➢ MooseFS ➢ Lustre ➢ CephFS ➢ StorPool (distributed block storage)
- 19. Network Network ➢ 2 separatenetworks ➢ public traffic (and backup for storage) ➢ storage & cluster traffic
- 20. Network Network ➢ 2 separatenetworks ➢ public traffic (and backup for storage) ➢ storage & cluster traffic ➢ DC redundancy is possible ➢ BGP ➢ Cloud Load Balancer ➢ DNS balancing (not advisable)
- 21. Storage - DB Storage- DB ➢ Replicated or Distributed DB ➢ MySQL or PostgreSQL with Master-Slave ➢ Redis or Cassandra for NoSQL ➢ etcd for distributed DB
- 22. Centralized vs. DistributedStorage vs. Synchronized Synchronized
- 23. Centralized Centralized ➢ exported overNFS ➢ block dev over: FC/iSCSI/FCoE/NVMe-oF/NVMe-oRDMA
- 24.
- 25. Distributed Distributed ➢ DRBD+OCFS2 (small) ➢GlusterFS/MooseFS (slow but scalable) ➢ OrangeFS ➢ StorPool ➢ Ceph (but requires big cluster)
- 26. Synchronized Synchronized ➢ rsync ➢ osync ➢bsync ➢ unison ➢ unisync ➢ syncthing
- 27. Decision making Decision making ➢Hashicorp Nomad ➢ Kubernetes StatefulSets ➢ Corosync + Pacemaker
- 28.
- 29.