Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
OmidEfficient Transaction Management and  Incremental Processing for HBase                                                ...
About me§ Research engineer at Yahoo!§ Long term Omid committer§ Apache S4 committer§ Contributor to: • ZooKeeper • HB...
Motivation
Motivation§ Traditional DBMS:            § NoSQL stores: • SQL                            • SQL • Consistent            ...
Motivation§ Lower latency • MapReduce latencies measured in hours • Usually low latency systems don’t provide strong   co...
Incremental Processing§ Small, consistent updates to a shared dataset§ Useful when: • Very large datasets • Consistent u...
Why Incremental Processing        Offline                    Online      MapReduce             Incremental Processing     ...
Transactions on HBase
Omid§ ‘Hope’ in Persian§ Optimistic Concurrency Control system • Lock free • Aborts transactions at commit time§ Implem...
Snapshot Isolation§ Each transaction reads from its own snapshot§ Snapshot: • Immutable • Identified by creation time • ...
Overlapping transactionsTransaction 1Transaction 2Transaction 3Transaction 4                                Time  § Trans...
Simple API§ Based on Java Transaction API and HBase API§ TransactionManager:     Transaction begin();     void commit(Tr...
Omid Architecture
Architecture1.  Centralized server2.  Transactional metadata replicated to clients3.  Store transaction ids in HBase times...
Omid Performance§ Centralized server = bottleneck? • Focus on good performance • In our experiments, it never became the ...
Metadata Replication§ Transactional metadata is replicated to clients§ Clients guarantee Snapshot Isolation • Ignore dat...
Fault Tolerance§ Omid uses BookKeeper for fault tolerance • BookKeeper is a Distributed Write Ahead Log§ Before answerin...
Fault Tolerance Overhead§ Omid batches writes to BookKeeper • Write every 5 ms or when the batch > 1 KB§ Recovery: reads...
Example Application   TF-IDF on Tweets
TF-IDF§ Term Frequency – Inverse Document Frequency • How important is a word to a document • Useful for search engines§...
TF-IDF on Tweets§ Given a set of words • Return relevant HashTags§ Document • collection of tweets with same hashtag§ U...
Implementation§ Read tweets’ stream, put them in queue§ Workers process each tweet in parallel • One transaction per twe...
Problems§ Hard to distribute load to other machines§ Complex processing cause big transactions • More likely to conflict...
Future Work
Future work§ Framework for Incremental Processing • Simpler API • Trigger-based, easy to decompose operations • Auto scal...
Contributors       Ben Reed       Flavio Junqueira       Francisco Pérez-Sorrosal       Ivan Kelly       Matthieu Morel   ...
Questions?
Upcoming SlideShare
Loading in …5
×

Omid Efficient Transaction Mgmt and Processing for HBase

2,397 views

Published on

Published in: Technology

Omid Efficient Transaction Mgmt and Processing for HBase

  1. 1. OmidEfficient Transaction Management and Incremental Processing for HBase Daniel Gómez Ferro 21/03/2013 Copyright © 2013 Yahoo! All rights reserved. No reproduction or distribution allowed without express written permission.
  2. 2. About me§ Research engineer at Yahoo!§ Long term Omid committer§ Apache S4 committer§ Contributor to: • ZooKeeper • HBase
  3. 3. Motivation
  4. 4. Motivation§ Traditional DBMS: § NoSQL stores: • SQL • SQL • Consistent • Consistent • Scalable • Scalable § Some use cases require • Scalability • Consistent updates
  5. 5. Motivation§ Lower latency • MapReduce latencies measured in hours • Usually low latency systems don’t provide strong consistency guarantees§ Requested feature • Support for transactions in HBase is a recurrent request§ Percolator • Google implemented transactions on BigTable
  6. 6. Incremental Processing§ Small, consistent updates to a shared dataset§ Useful when: • Very large datasets • Consistent updates • Incremental changes
  7. 7. Why Incremental Processing Offline Online MapReduce Incremental Processing Fault tolerance Fault tolerance Scalability Scalability Shared state State0 State1 Shared State Using Percolator, Google dramatically reduced crawl-to-index time
  8. 8. Transactions on HBase
  9. 9. Omid§ ‘Hope’ in Persian§ Optimistic Concurrency Control system • Lock free • Aborts transactions at commit time§ Implements Snapshot Isolation§ Low overhead§ https://github.com/yahoo/omid
  10. 10. Snapshot Isolation§ Each transaction reads from its own snapshot§ Snapshot: • Immutable • Identified by creation time • Contains all values committed before creation time§ Transactions conflict if two conditions apply: A) Write to the same row B) Overlap in time
  11. 11. Overlapping transactionsTransaction 1Transaction 2Transaction 3Transaction 4 Time § Transaction 2 could conflict with 1 or 3 • If they write to the same row § Transaction 4 doesn’t conflict • No overlapping transactions
  12. 12. Simple API§ Based on Java Transaction API and HBase API§ TransactionManager: Transaction begin(); void commit(Transaction t) throws RollbackException; void rollback(Transaction t);§ TTable: Result get(Transaction t, Get g); void put(Transaction t, Put p); ResultScanner getScanner(Transaction t, Scan s);
  13. 13. Omid Architecture
  14. 14. Architecture1.  Centralized server2.  Transactional metadata replicated to clients3.  Store transaction ids in HBase timestamps HBase Client HBase Client S Omid HBase Client Clients O (2) HBase Status Oracle (SO) HBase Region HBase Region Servers HBase Region Servers Region Servers (1) Servers (3)
  15. 15. Omid Performance§ Centralized server = bottleneck? • Focus on good performance • In our experiments, it never became the bottleneck 30 2 rows 4 rows 25 8 rows 16 rows 32 rows 20 128 rows Latency in ms 512 rows 15 10 5 0 1K 10K 100K Throughput in TPS
  16. 16. Metadata Replication§ Transactional metadata is replicated to clients§ Clients guarantee Snapshot Isolation • Ignore data not in their snapshot • Talk directly to HBase • Conflicts resolved by the Status Oracle§ Expensive, but scalable up to 1000 clients
  17. 17. Fault Tolerance§ Omid uses BookKeeper for fault tolerance • BookKeeper is a Distributed Write Ahead Log§ Before answering a commit request • Log it to BookKeeper asynchronously • Wait for a reply • Notify the client§ If the Status Oracle crashes • Recover the state from BookKeeper
  18. 18. Fault Tolerance Overhead§ Omid batches writes to BookKeeper • Write every 5 ms or when the batch > 1 KB§ Recovery: reads log’s tail (bounded time) Downtime < 25 s 18000 Throughput in TPS 15000 12000 9000 6000 3000 0 5900 5950 6000 6050 6100 Time in s
  19. 19. Example Application TF-IDF on Tweets
  20. 20. TF-IDF§ Term Frequency – Inverse Document Frequency • How important is a word to a document • Useful for search engines§ Given a set of words (query) • Return documents with highest TF-IDF
  21. 21. TF-IDF on Tweets§ Given a set of words • Return relevant HashTags§ Document • collection of tweets with same hashtag§ Update the index incrementally
  22. 22. Implementation§ Read tweets’ stream, put them in queue§ Workers process each tweet in parallel • One transaction per tweet • Update frequencies consistently • In case of abort, retry§ Queries have a consistent view of the database
  23. 23. Problems§ Hard to distribute load to other machines§ Complex processing cause big transactions • More likely to conflict • More expensive to retry§ API is too low level for data processing
  24. 24. Future Work
  25. 25. Future work§ Framework for Incremental Processing • Simpler API • Trigger-based, easy to decompose operations • Auto scalable§ Integrate Omid with other Data Stores
  26. 26. Contributors Ben Reed Flavio Junqueira Francisco Pérez-Sorrosal Ivan Kelly Matthieu Morel Maysam Yabandeh
  27. 27. Questions?

×