Gizzard, DAL and more

4,406 views

Published on

About Twitter's distribute db sharding framework, and Data Access Layer from iMobile, and a Discuss

0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,406
On SlideShare
0
From Embeds
0
Number of Embeds
2,597
Actions
Shares
0
Downloads
38
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Gizzard, DAL and more

  1. 1. gizzard, DAL and more @tangfl 2011.2.22
  2. 2. gizzard 1• sharding framework• extract from FlockDB• written in Java/Scala
  3. 3. gizzard 2• middle ware : • like mysql-proxy ?• any backends • like mytrigger ? • adapter ?
  4. 4. gizzard 3• rules: forwarding table • maps key ranges to partitions• replication tree
  5. 5. gizzard 4• eventually consistent • idempotent : f(f(n)) = f(n) • commutative : f(a) + f(b) = f(b) + f(a) • retry queue
  6. 6. gizzard 5• gracefully failures • read : route to remain ones • write : make high priority to remain replicas if any in a shard are unavailable • retry queue
  7. 7. gizzard 6• gracefully migrations • replica tree • WriteOnly shard
  8. 8. gizzard 7• summary • nothing new • Get work done
  9. 9. DAL 1• Data Access Layer• from iMobile• by longker, simaliu and fulin• Not only sharding
  10. 10. DAL 2• data table, hash by pri id, less index• index tables, hash by any “where field” on demand• front php devs : I don’t care !
  11. 11. DAL 3• insert / update / delete : auto update index table• query : analyze sql, choose best index table
  12. 12. DAL 4• build-in cache : php don’t care about cache anymore
  13. 13. DAL 5• any backend • treat Lucene just like mysql : select field from table where condition• no replica • backend do the job
  14. 14. DAL 6• summary • nothing new • Get work done
  15. 15. more• Discuss • fans_all list in redis • vary from 0 to 500w • single hash at most 5000 • auto resharding • async
  16. 16. Thanks• @tangfl http://t.sina.com.cn• http://blog.fulin.org

×