Amazon`s Dynamo Design and Implementation Key learning`s by using it in production system

1. Amazon`s
Presented By
Sarang Metkar
Dynamo: Highly Available Key-value Store
Giuseppe DeCandia
Deniz Hastorun
Madan Jampani
Gunavardhan Kakulapati
and others

2. Introduction
• Highly available and scalable distributed data store
• Flexible key – value data model

3. Key-Value Data Model
• Simple Key-Value pairs
• Table - collection of items
• Item - collections of attributes

4. Introduction
• Highly available and scalable distributed data store
• Flexible key – value data model
• Fast performance with seamless scalability
• Eventually consistent
• Decentralized system

5. Motivation
• ‘Always On’ experience to large customer base
• Reduce impact of failure without compromising
performance
• Diverse applications with different storage and
data access requirement
• Configurable to achieve stringent SLAs

6. SLA requirements
• Decentralized service
oriented architecture
• Multiple dependencies
hence tight constraints
• 99.9th percentile SLA
measurement
Reference : Dynamo: Amazon’s Highly Available Key-value Store : Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Alex Pilchin, Peter Vosshall
and Werner Vogels

7. Design Considerations - Consistency
• Availability using optimistic replication - eventual
consistency
• Challenges in conflict resolution
• When to resolve ?
- Always writable requirement
• Who resolves?
- Application assisted
- Data store`s “last write wins” policy

8. Other Design Considerations
• Incremental Scalability
• Symmetry
• Decentralization
• Heterogeneity

9. System Interface
• Object storage and access
• get(key)
- Locate object replicas
- Return single or list of objects
• put(key, context, object)
- Determine location of replica
- Context for conflict resolution

10. Partition and Replication
• Consistent hashing for load and data
distribution
• Less impact of addition or deletion of
nodes
• Virtual nodes account for heterogeneity
• Coordinator node stores preference list

11. Eventual Consistency
• Asynchronous updates of replicas
• Versioning, based on vector clocks
• Reconciliation
• Syntactic reconciliation
• Semantic reconciliation
• Sloppy quorum like consistency
protocol
• Configurable R , W and N
[ R + W > N ]
Reference : Dynamo: Amazon’s Highly Available Key-value Store : Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Alex Pilchin, Peter Vosshall
and Werner Vogels

12. Handling failures
• High Availability and Durability requirement
• Hinted handoff – temporary failures
• Replica synchronization – permanent failures
• Merkle Trees
- Less data transfer and faster replication
- One for each key range on node
- Recalculation of tree on key range changes

13. Membership and Failure Detection
• Manual addition or removal of nodes
• Gossip based protocol to reconcile membership
changes
• Partitioning and node-to-token sets mapping
information propagation
• Seeds avoids logical partitioning
• Decentralized failure detection

14. Implementation
• Local persistence engine
• Application specific
• Pluggable
• Request coordination
• Read/write request execution
• Read repair
• ‘Read-your-writes’ consistency
• Java NIO channel
• Membership and failure detection

15. Key Learnings - 1
• Common (N,R,W) configuration – (3, 2, 2)
• Balancing Performance and Durability
• Buffering of write operations
• Durable write

16. Key Learnings [cont`d]
• Uniform load distribution
• More load imbalance for low load
• Q/S tokens per node, equal-sized partition
• Faster bootstrapping and recovery
• Ease of archival

17. Key Learnings [cont`d]
• Divergent Versions
• Failures in system
• Concurrent writes to single object by multiple nodes
• Client driven coordination
• Request coordination at client
• Pull membership information
• Reduces latency
• Admission Control mechanism for background
tasks

18. Related Work
• Peer to Peer systems
• Unstructured peer-to-peer network
• Gnutella [1]
• Freenet [2]
• Structure peer-to-peer network
• Oceanstore [3]
• Beehive [4]
• Distributed File Systems and Databases
• Google File System [5]
• Bayou [6]

19. Conclusion
• Application specific configuration for availability,
durability, performance and consistency
• Evaluation of different techniques to build highly
available system
• Use of eventually consistent storage system in
production
• Tuning of various techniques to meet strict production
performance requirements

20. References
• [1] http://www.gnutella.org/
• [2] http://freenetproject.org/
• [3] Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D.,
Gummadi, R., Rhea, S., Weatherspoon, H., Wells, C., and Zhao, B. 2000.
OceanStore: an architecture for global-scale persistent storage.
• [4] Ghemawat, S., Gobioff, H., and Leung, S. 2003. The Google file system. In
Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles
• [5] Ramasubramanian, V., and Sirer, E. G. Beehive: O(1)lookup performance for
power-law query distributions in peer-to-peer overlays.
• [6] Terry, D. B., Theimer, M. M., Petersen, K., Demers, A. J., Spreitzer, M. J., and
Hauser, C. H. 1995. Managing update conflicts in Bayou, a weakly connected
replicated storage system.

21. Thank You