This document proposes ThriftyPaxos, a state machine replication protocol that provides cheap and available replication through two techniques: (1) on-demand instantiation that activates backup replicas only when needed, and (2) lazy recovery that performs state transfer in the background without halting the system. The evaluation shows that ThriftyPaxos reduces replication costs compared to standard Paxos, and maintains availability during state transfers unlike CheapBFT, making it suitable for large-scale systems.

1. Cheap and Available State Machine
Replication
Rong Shi, Yang Wang(The Ohio State University)
ATC 2016
Yongrae Jo (20172938)

2. Active replica
Client
replica 1
replica 2
replica 4
replica 3
replica 5
replica 6
replica 7
Backup
In active replica / backup
replica organization, CFT

3. Active replica
Client
replica 1
replica 2
replica 4
replica 3
replica 5
replica 6
replica 7
Backup
In active replica / backup
replica organization, CFT

4. State transfer
Active replica
Client
replica 1
replica 2
replica 4
replica 3
replica 5
replica 6
replica 7
Backup
In active replica / backup
replica organization, CFT

5. Activate backup replica
(state transfer)
Request System not available
during state transfer
Active replica
Client
replica 1
replica 2
replica 4
replica 3
replica 5
replica 6
replica 7
Backup
In active replica / backup
replica organization, CFT

6. Completes
state transfer
Request System now available
after state transfer
Active replica
Client
replica 1
replica 2
replica 4
replica 3
replica 5
replica 6
replica 7
Backup
In active replica / backup
replica organization, CFT

7. Cheap
● Reducing replication cost
– 2f + 1 replicas for f crash faults
● f+1 active replicas participating in agreement or
execution stage
– 3f + 1 replicas for f arbitrary faults
● 2f + 1 active replicas participating in agreement or
execution stage
● Why needed?
– Large data center where up to millions of
machines may need to be replicated

8. Available
● Active / Backup replica scheme,
– When a active replica fails, backup replica
needs to be active one
– Then, backup replica needs to get state
transfer from active one
– System is not available during state transfer
– For applications with a large state, the system
can be halted for long
● Providing system availability under state transfer
with active / backup replica scheme

9. On-demand instantiation
● Activates a subset of replicas first and activates
backup one when active one fails
● Different from Active / Passive replica
scheme(CheapBFT)
– Passive replica simply updates result
● No need to transfer state when faulty replica
occured
– Backup replica does nothing until it is activated

10. Lazy recovery
● Realizing system availability during state transfer using
adaptive recovery
– Rebuilding a backup replica’s state in the background
without halting the system
● Adaptive recovery
– Coordinate recovery speed with transferred state by
periodically checking the progress
– It compares (fetched data / total data) and (elapsed
time / deadline) to coordinate recovery speed
– During adaptive recovery, client’s request is also
processed in parallel which incurs competition for
resources (e.g. network and disk bandwidth)
●
Lazy recovery is possible only when agreement and
execution are logically separated

11. On-demand instantiation
Unavailability
under recovery
Lazy recovery
SMR with
Low replication cost
SMR with
Low replication cost
& High availability
SMR
SMR with
Low replication cost

12. ThriftyPaxos
● Applying on-demand instantiation and lazy
recovery to standard Paxos
● ThriftyPaxos mostly resembles standard Paxos
– Normal case op., leader change protocol
– Not new and complete protocol
● ThriftyPaxos activates f +1 replicas and reserves f
replicas as backup
– Standard Paxos activates 2f + 1 replicas
● Same correctness guarantee as Paxos with f
fewer replicas

13. Evaluation
● RemoteHashMap
– An application that allows clients to get and set
key value pairs on a remote server
– It can change system parameters (e.g. request
size) arbitrarily to test different aspects of
ThriftyPaxos.
● 7 Dell R220 machines, each with 8 cores, 16GB of
memory, and two hard drivers
● Store RemoteHashMap’s data on one hard drive
and store agreement nodes’ data on the other
hard drive
● 5 minutes for triggering recovery after detecting
crash

14. Evaluation
Key value performance
v: value size
f=1 and v=512b

15. Availability of ThriftyPaxos, standard Paxos, and Cheap Paxos
Not available
f+1 active replica
2f+1 active replica
f+1 active replica,
on-demand instantiation
System is still available

16. Backup agreement node needs only few messages to recover
Backup execution node needs
recovery(state transfer) Backup execution node
replays the logs
Execution node depends on previous request to execute next request
→ It needs request logs(state transfer) and replay logs
Agreement node doesn’t require previous request to determine next request
(memory less property)

17. Less resource
allocated
to recovery task
Adaptive recovery
5 mins after crash
22240 req/s
74690 req/s
99339 req/s

18. Conclusion
● By combining on-demand instantiation and lazy
recovery, cheap and available state machine
replication protocol is achieved
● No need to invent new protocols
– By carefully looking at exsisting protocols,
ThriftyPaxos slightly changes standard Paxos
to provide on-demand instantiation and lazy
recovery

19. Thanks

20. Backup slides

21. Latency critical task
(e.g. client request)
Latency flexible task
(e.g. garbage collection)