Data System - Human fault-tolerance

1,199 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,199
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Data System - Human fault-tolerance

  1. 1. Human fault-tolerance Nathan Marz Twitter 1
  2. 2. What is a data system? A system that manages the storage and querying of data
  3. 3. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years
  4. 4. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist
  5. 5. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist, every hardware failure
  6. 6. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist, every hardware failure, and every human mistake ever made
  7. 7. +
  8. 8. We don’t know how to make perfect software
  9. 9. Paxos tunable consistencyreplication active anti-entropyCRC’s merkle treesBASE read repairquorum consistent hashing Machine fault-tolerance
  10. 10. The worst consequence isdata loss or data corruption
  11. 11. As long as an error doesn’t lose or corrupt good data,you can fix what went wrong
  12. 12. This brings us to mutability
  13. 13. Mutability
  14. 14. The U and D in CRUD
  15. 15. Mutable systems inherentlylack human fault-tolerance
  16. 16. Very easy for a mistake to corrupt or lose data
  17. 17. Immutability
  18. 18. Immutability• An immutable system captures a historical record of events• Each event happens at a particular time and is always true
  19. 19. Capturing change with mutable datamodel Person Location Person Location Sally Philadelphia Sally New York Bob Chicago Bob Chicago Sally moves to New York
  20. 20. Capturing change with immutabledata modelPerson Location Time Person Location Time Sally Philadelphia 1318358351 Sally Philadelphia 1318358351 Bob Chicago 1327928370 Bob Chicago 1327928370 Sally New York 1338469380 Sally moves to New York
  21. 21. Immutability greatly restricts the range of errors that can cause data loss or data corruption
  22. 22. Vastly more human fault-tolerant
  23. 23. Plus much easier to reason about systems based on immutability
  24. 24. What can we conclude?
  25. 25. Your source of truth should be immutable
  26. 26. Mutable Application database SOURCE OF TRUTHRather than build systems like this...
  27. 27. MySQL Cassandra HBase Application Voldemort Riak ... SOURCE OF TRUTHRather than build systems like this...
  28. 28. Immutable View Application dataSOURCE OF TRUTH ...build them like this
  29. 29. Cassandra HBase Voldemort HDFS Application Riak ElephantDB ...SOURCE OF TRUTH ...build them like this
  30. 30. Long live immutability (forgive the pun)

×