Download to read offline
Uploaded byEnsar Basri Kahveci
java.util.concurrent for Distributed Coordination, GeeCON Krakow 2019
The document discusses distributed coordination in systems, detailing APIs and frameworks such as Google Chubby, Apache Zookeeper, and etcd. It introduces Java's concurrency utilities as a high-level approach for distributed coordination tasks, emphasizing the importance of operational simplicity and avoiding DIY implementations. The document also includes demos and future plans regarding configuration management, leader election, and distributed locking solutions.
More Related Content
Similar to java.util.concurrent for Distributed Coordination, GeeCON Krakow 2019
java.util.concurrent for Distributed Coordination, GeeCON Krakow 2019
- 1. java.util.concurrent for distributed coordination EnsarBasri Kahveci Distributed Systems Engineer @ Hazelcast @ metanet Kraków, 15-17 May 2019
- 2.
- 3. @metanet Agenda What is distributedcoordination? How distributed coordination APIs evolved over time? java.util.concurrent.* for distributed coordination Demo on Hazelcast IMDG 3.12
- 4.
- 5. @metanet Distributed Coordination Leader election Synchronization Groupmembership Configuration and metadata management
- 6.
- 7. @metanet Consensus algorithms underthe hood CP with respect to CAP Deployed as a central repository APIs for coordination tasks Distributed Coordination Systems
- 8.
- 9. Google Chubby (Paxos) ApacheZooKeeper (ZAB) @metanet
- 10. Google Chubby (Paxos) ApacheZooKeeper (ZAB) etcd (Raft) @metanet
- 11.
- 12.
- 13. @metanet A Simple LockingRecipe for ZooKeeper 1. create an ephemeral znode “/lock” 2. if success, enter to the critical section 3. else, register a watch on “/lock” 4. when the watch is notified, i.e., the lock is released, retry step #1
- 14. @metanet Chubby Locking APIs Recipes “Friendsdon't let friends write ZK recipes.” Apache Curator Tech Notes #6 Leader election and distributed lock primitives ZooKeeper etcd
- 15. @metanet High-level APIs A low-levelfile-system / KV store API is - easy to misuse, - not suitable for all coordination tasks. High-level APIs minimise guesswork and development effort. java.util.concurrent.* in JDK
- 16.
- 17. Google Chubby (Paxos) ApacheZooKeeper (ZAB) etcd (Raft) Hazelcast IMDG 3.12 java.util.concurrent on top of Raft @metanet
- 18. @metanet An Opinionated &High-Level Framework IAtomicLong, IAtomicReference, ICountDownLatch, ISemaphore, FencedLock Well-defined failure semantics CP with respect to CAP DIY-style tested with Jepsen
- 19. @metanet Understandability as aprimary goal Handles crash failures and network failures. Operational as long as the majority is up. Runtime concerns (snapshotting, dynamic membership) Performance optimizations (fast reads, batching) https://raft.github.io Why Raft?
- 20. @metanet A leader iselected among the nodes. The leader replicates ops to the followers. All nodes run the ops in the same order. Replicated State Machines
- 21. @metanet CP Subsystem Minimal configuration CPprimitives and AP data structures in the same cluster Dynamic clustering programmatically or via REST API
- 22. @metanet Horizontal Scalability CP primitivesrun in CP groups. Each CP group runs the Raft algorithm independently (Multi-Raft). CP groups can be distributed to CP members.
- 23.
- 24.
- 25. @metanet FencedLock Linearizable distributed implof java.util.concurrent.locks.Lock Suitable for both fine-grained and coarse-grained locking
- 26. @metanet CP Sessions A sessionstarts on the first lock / semaphore request. Session heartbeats are periodically committed in the background. If no heartbeat for some time (session TTL), the session is closed. Auto-release mechanism for FencedLock and ISemaphore
- 27. @metanet DEMO #2: AddingRedundancy We use FencedLock for leader election.
- 28. @metanet CP sessions offera trade-off between safety and liveness.
- 29. @metanet DEMO #3: Fencing-offStale Lock Holders “How to do distributed locking” “Distributed locks are dead; long live distributed locks!”
- 30. @metanet Recap Avoid writing yourown implementations for coordination. High-level APIs minimise guesswork and development effort. java.util.concurrent.* FTW! Operational simplicity matters. Dynamic clustering Horizontal scalability
- 31. @metanet Future Plans KV Store EventListeners Disk persistence Tooling
- 32. @metanet Resources https://github.com/metanet/juc-talk (demos) Hazelcast IMDGDocs CP Subsystem Code Samples https://hazelcast.com/blog/author/ensarbasri Hazelcast IMDG 3.12
- 33. Dzięki! Ensar Basri Kahveci @metanet slido.com#geecon2019 Kraków, 15-17 May 2019