Все чаще хочется иметь под рукой удобный набор инструментов для решения задач, возникающих при проектировании и разработке распределенных приложений. В своем докладе я расскажу про то, какие средства предлагает фреймворк Atomix для гарантии отказоустойчивости ваших приложений, а также координации и репликации ресурсов. Предварительно я расскажу про алгоритм консенсуса Raft и то, какую важную роль он играет в Atomix.
2. 2
About me
• Software Engineer @ EPAM
• HPC Engineer and Researcher @ ITMO University
3. 3
Atomix
• Distributed data-structure/coordination toolkit written in Java
• Provides a collection of asynchronous APIs for state sharing and solving a variety of
common distributed systems problems
• Reactive thus asynchronous and event-driven
• Provides strong consistency using its own Raft consensus algorithm implementation
4. 4
What I will cover
• Raft algorithm
• Consensus
• State machines
• Log replication
• Atomix APIs
• Distributed data-structures API
• Coordination and messaging API
• Copycat
• Raft implementation
• State machine API
5. 5
Consensus
• Agreement on shared state
• Autonomous recovery from server failures
– Minority of servers fail: no problem
– Majority fail: lose availability, retain consistency
Servers
6. 6
What is the purpose of consensus?
• Key to building consistent storage systems
• Top-level system configuration
– Which server is the master?
– What shards exist in my storage system?
– Which servers store shard X?
• Sometimes used to replicate entire storage state
7. 7
Raft
x3 y2 x1 z6
Log
Consensus
Module
State
Machine
Log
Consensus
Module
State
Machine
Log
Consensus
Module
State
Machine
Servers
Clients
x 1
y 2
z 6
x3 y2 x1 z6
x 1
y 2
z 6
x3 y2 x1 z6
x 1
y 2
z 6
z6
8. 8
Raft terms
1. Leader election
– Select one of the servers to act as leader
– Detect crashes, choose new leader
– Only elect leaders with all committed entries in their logs
2. Log replication (normal operation)
– Leader takes commands from clients, appends them to its log
– Leader replicates its log to other servers (overwriting inconsistencies)
Term 1 Term 2 Term 3 Term 4 Term 5
time
Elections Normal OperationSplit Vote
9. 9
Distributed data-structures
• Drop-in replacements for Java Collections
• Similar to Hazelcast distributed collections
• Strong consistency over availability
• Asynchronous with CompletableFuture
13. 13
Coordination and messaging API
• Locks
• Group membership management and leader election listening
• Direct messaging within groups
• Publish-subscribe
• Request-reply
17. 17
Request-reply
DistributedGroup group = atomix.getGroup("group").join();
Member member = group.member("foo");
MessageProducer.Options options = new MessageProducer.Options()
.withExecution(Execution.REQUEST_REPLY);
MessageProducer<String> producer = member.messaging().producer("hello");
producer.send("Hello world!").thenAccept(reply -> { System.out.println(reply); });
DistributedGroup group = atomix.getGroup("group").join();
LocalMember localMember = group.join().join();
MessageConsumer<String> consumer = localMember.messaging().consumer("hello");
consumer.onMessage(message -> {
if (message.body().equals("Hello world!")) {
messages.reply("Hello world back!");
}
});
18. 18
What can I do with that?
• Use as the replacement for Zookeeper or etcd with High Level API
• Reliable messaging
• Simply make your Java collections distributed
• Resilient distributed caches
• Create any kind of distributed resource with fault-tolerance and strong consistency
through custom Atomix resource implementation
20. 20
Command pattern
public class PutCommand extends Command<String> {
String key
String value
}
public class GetQuery extends Query<String> {
String key
}
21. 21
State machine API
public class MapStateMachine extends StateMachine {
private Map<String, String> map = HashMap<>();
public String put(Commit<PutCommand> commit) {
try {
map.put(commit.operation().key(), commit.operation().value());
} finally {
commit.close();
}
}
public String get(Commit<GetQuery> commit) {
try {
return map.get(commit.operation().key());
} finally {
commit.close();
}
}
}
22. 22
Copycat Server API
Address address = new Address(“localhost", 5000);
Collection<Address> cluster = Arrays.asList(
new Address("192.168.0.1", 8700),
new Address("192.168.0.2", 8700),
new Address("192.168.0.3", 8700)
);
CopycatServer server = CopycatServer.builder(address)
.withStateMachine(MyStateMachine::new)
.build();
server.bootstrap(cluster).join();
23. 23
Copycat Client API
CopycatClient client = CopycatClient.builder()
.withTransport(new NettyTransport())
.withServerSelectionStrategy(ServerSelectionStrategies.FOLLOWERS)
.withConnectionStrategy(ConnectionStrategies.EXPONENTIAL_BACKOFF)
.build();
Collection<Address> cluster = Arrays.asList(
new Address("192.168.0.1", 8700),
new Address("192.168.0.2", 8700),
new Address("192.168.0.3", 8700)
);
client.connect(cluster).join();
client.submit(new PutCommand("foo", "Hello world!")).thenRun(() -> {
String value = client.submit(new GetQuery("foo")).join();
});
24. 24
Why would I need that?
• If Atomix is too high level for you or so much for your task
• If you have some rather sophisticated resource you want to make consistent
• If you just want to practice or implement your ideas
25. 25
In the end
• Atomix provides you with very simple distributed programming interface which allows
you to solve common distributed system problems
• Raft consensus algorithm gives you strong consistency (but over availability)
• Copycat provides you with low level state machine API which could become the
barebone for your distributed resource