Apache Ratis - In Search of a Usable Raft Library

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

© Hortonworks Inc. 2017
Apache Ratis
In Search of a Usable Raft Library
Tsz-Wo Nicholas Sze
11/15/2017
Brown Bag Lunch Tal...

© Hortonworks Inc. 2017
About Me
• Tsz-Wo Nicholas Sze, Ph.D.
– Software Engineer at Hortonworks
– PMC member/Committer of...

© Hortonworks Inc. 2017
Agenda
• A brief introduction of RAFT
• Apache Ratis
– Features and use cases
– Demo
• Project Sta...

© Hortonworks Inc. 2017
Consensus
• What is consensus?
– Multiple servers to agree a value
• Typical use cases
– Log repli...

© Hortonworks Inc. 2017
Consensus Algorithms
• Paxos (1990)
– Work but hard to understand
– Hard to implement (correctly)
...

© Hortonworks Inc. 2017
Raft Basic
• Leader Election
– Servers are started as a Follower
– Randomly timeout to become Cand...

© Hortonworks Inc. 2017
Raft Library
• Our Motations
– Use Raft in Ozone
• “In Search of a Usable Raft Library”
– A long l...

© Hortonworks Inc. 2017
Ratis: Standard Raft Features
• Leader Election + Log Replication
– Automatically elect a leader a...

© Hortonworks Inc. 2017
Ratis: Pluggability
• Pluggable state machine
– Application must define its state machine
– Exampl...

© Hortonworks Inc. 2017
Data Intensive Applications
• In Raft,
– All transactions and the data are written in the log
– No...

© Hortonworks Inc. 2017
Ratis: Asynchronous APIs
• Using gRPC bi-directional stream API
– Netty and Hadoop RPC can support...

© Hortonworks Inc. 2017
General Ratis Use Cases
• You already have a service running on a single server.
• You want to:
– ...

© Hortonworks Inc. 2017
Ozone/HDFS Use Cases
• Replicating open containers
– HDFS-11519
• Ozone: Implement XceiverServerSp...

© Hortonworks Inc. 2017
Example: ArithmeticStateMachine
• Maintain a variable map (variable -> value)
– Users define the v...

© Hortonworks Inc. 2017
Demo: ArithmeticStateMachine
• Pythagorean
– Put a = 3 and b = 4
– Find c
• Special thanks to Mart...

© Hortonworks Inc. 2017
Gauss-Legendre
• A.k.a the arithmetic–geometric mean method
• A fast algorithm to compute pi
Page ...

© Hortonworks Inc. 2017
Example: FileStore (RATIS-122)
• Maintain a file map (key -> file)
• Support only
– Read, Write, D...

© Hortonworks Inc. 2017
Ratis: Development Status
• A brief history
– 2016-03: Project started at Hortonworks
– 2016-04: F...

© Hortonworks Inc. 2017
Work in Progress
• Multi-Raft
– General idea: Allow a server to join multiple Raft groups (RATIS-9...

© Hortonworks Inc. 2017
Ratis: Future Works
• Performance
– Fix gRPC async bugs: RATIS-140 and RATIS-141
– Use FileStore a...

© Hortonworks Inc. 2017
Contributors
– Animesh Trivedi, Anu Engineer, Arpit Agarwal, Brent,
– Chen Liang, Chris Nauroth, D...

© Hortonworks Inc. 2017
Thank you!
Contributions are welcome!
• http://incubator.apache.org/projects/ratis.html
• dev@rati...

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

LinkedIn
Facebook
Twitter
Email

Replay

Top clipped slide

1 of 23 Ad

1 of 23 Ad