Your SlideShare is downloading. ×
Data System  - Human fault-tolerance
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data System - Human fault-tolerance

746

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
746
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Human fault-tolerance Nathan Marz Twitter 1
  • 2. What is a data system? A system that manages the storage and querying of data
  • 3. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years
  • 4. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist
  • 5. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist, every hardware failure
  • 6. What is a data system? A system that manages the storage and querying of data with a lifetime measured in years encompassing every version of the application to ever exist, every hardware failure, and every human mistake ever made
  • 7. +
  • 8. We don’t know how to make perfect software
  • 9. Paxos tunable consistencyreplication active anti-entropyCRC’s merkle treesBASE read repairquorum consistent hashing Machine fault-tolerance
  • 10. The worst consequence isdata loss or data corruption
  • 11. As long as an error doesn’t lose or corrupt good data,you can fix what went wrong
  • 12. This brings us to mutability
  • 13. Mutability
  • 14. The U and D in CRUD
  • 15. Mutable systems inherentlylack human fault-tolerance
  • 16. Very easy for a mistake to corrupt or lose data
  • 17. Immutability
  • 18. Immutability• An immutable system captures a historical record of events• Each event happens at a particular time and is always true
  • 19. Capturing change with mutable datamodel Person Location Person Location Sally Philadelphia Sally New York Bob Chicago Bob Chicago Sally moves to New York
  • 20. Capturing change with immutabledata modelPerson Location Time Person Location Time Sally Philadelphia 1318358351 Sally Philadelphia 1318358351 Bob Chicago 1327928370 Bob Chicago 1327928370 Sally New York 1338469380 Sally moves to New York
  • 21. Immutability greatly restricts the range of errors that can cause data loss or data corruption
  • 22. Vastly more human fault-tolerant
  • 23. Plus much easier to reason about systems based on immutability
  • 24. What can we conclude?
  • 25. Your source of truth should be immutable
  • 26. Mutable Application database SOURCE OF TRUTHRather than build systems like this...
  • 27. MySQL Cassandra HBase Application Voldemort Riak ... SOURCE OF TRUTHRather than build systems like this...
  • 28. Immutable View Application dataSOURCE OF TRUTH ...build them like this
  • 29. Cassandra HBase Voldemort HDFS Application Riak ElephantDB ...SOURCE OF TRUTH ...build them like this
  • 30. Long live immutability (forgive the pun)

×