Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BigTable PreReading

431 views

Published on

  • Be the first to comment

  • Be the first to like this

BigTable PreReading

  1. 1. BIGTABLE READING 2013.04.02 xielun.szd@alipay.com
  2. 2. OutLine  Introduction  Data Model  API  Building Blocks  Implementation  Refinements  Performance Evaluation  Real Applications  Lessons  Related Work  Conclusions
  3. 3. OutLine  Introduction  Data Model  API  Building Blocks  Implementation  Refinements  Performance Evaluation  Real Applications  Lessons  Related Work  Conclusions
  4. 4. Introduction  Why Google Need BigTable ?  Why BigTable Need LSM ?
  5. 5. Data Model  BigTable Is A Sparse Sorted Map  Key: <Row, Column Family, Qualifier, Timestamp>
  6. 6. API  How to Support Single-Row transaction  Atomic Read-modify-write  Using API for MapReduce  Input source & Output target
  7. 7. Building Blocks  GFS  SSTable Data File  Commit Log (WAL & Redo Log)  Chubby  Distributed Lock Service  Root Tablet Location  Tablet server Manager  Schema & ACL Metadata  Why GFS not use chubby For Master?  If Chubby not exist, how to design the bigtable?
  8. 8. Implementations  Client  How To Protect Lighted Master ?  Master  Tablet server Manager  Tablet Assignment  Garbage Collection  Schema Change  Do We Need Slave Master For Availability?  Tablet Server  Reader & Writer
  9. 9. Implementations  Tablet Locations
  10. 10. Implementations  Why select Three-level hierarchy?  Why Persistent the location info?  Why analogous as B+tree?  The Most RPC times during One simple Query?
  11. 11. Implementations  Tablet Assignment  Consistency Guaranty ?  Tablet Server Offline detection Procedure  Master Recovery Procedure ?  Master Memory != Metadata Table  Metadata Table Schema Design?
  12. 12. Implementations  Tablet Serving  Read & Write & Recovery Procedure
  13. 13. Implementations  Compactions  Three Type Compactions Differences  If client Write 1GB data, How much data in GFS?
  14. 14. Refinements  How To Reduce IO?  Column Store  Compression  Block Size  Caching  Caching Consistency  Bloom Filter  Why Must Using Bloom Filter For LSM?  Group Commit  Prons Vs Cons
  15. 15. Refinements  Log  GFS exist duplicated record?  What Happened when Tablet Server Crashes?  Tablet Recovery  Tablet Migrate  Compare with Tair & OB  Tablet Split  Tablet Merge  How can do this online?
  16. 16. Performance  Compare With Single Server Performance  Low Latency Vs High Through output  Why Performance/Server Decrease?
  17. 17. Performance  Scalability
  18. 18. Lessons  Vulnerable to many types of failure  RPC crc checksum  Removing assumption  Say No to adding new features  System-level Monitoring  Simple Design
  19. 19. Related Work  CAP in BigTable?  Sharing Nothing or Sharing Disk or Sharing Memory?  CStore or BigTable or OceanBase
  20. 20. Assignment  ALL Questions Above  CStore, Bloom Filter  Chubby, Megastore  Dynamo  Cassandra  Riak  Volmort  Redis cluster  Thanks & QA

×