The document summarizes Hypertable, an open source scalable database modeled after Bigtable. It discusses Hypertable's architecture, data representation, scaling abilities, query handling, features like namespaces, column families, access groups, and regular expression filtering. It also evaluates Hypertable's performance compared to HBase, finding it has advantages for certain workloads like random reads and writes. Upcoming releases will focus on automatic range balancing and an improved monitoring system.

1. Hypertable Doug Judd CEO, Hypertable, Inc.

2. High Performance, Open Source Scalable Database Modeled after Bigtable High Performance Implementation (C++) Project Started in March 2007 Runs on top of HDFS Thrift Interface for all popular languages Java PHP Ruby Python Perl, etc.

3. Hypertable Deployments

4. Architecture

5. Underlying Data Representation

6. Scaling (part I)

7. Scaling (part II)

8. Scaling (part III)

9. Request Routing

10. Query Handling

11. Features

12. Load data from HT to Hive and vice-versa Use Hive types Use Hive QL (joins, aggregations) Low latency data warehousing Uses Hypertable’s native MapReduce Input/Output format

13. Namespaces /development user tweet /testing user tweet /production /v1 user tweet /v2 user tweet

14. Column Family Options TTL=<t> “ time to live” Remove cells that are older than <t> MAX_VERSIONS=<n> Keep only most recent <n> cell versions

15. Access Groups Provides control over physical layout Row oriented Column oriented Hybrid Reduces I/O CREATE TABLE MyTable ( a, b, c, d, ACCESS GROUP first(a), ACCESS GROUP second (b, c, d) );

16. Regular Expression Filtering Google’s RE2 regular expression engine Extremely fast (up to 50X Java regex) Searches run in time linear in the size of the input Searches constrained to a fixed amount of memory Supported Searches: Row key Column qualifier Value SELECT CELLS tag:/(?i)(nosql|bigtable)/ FROM MyTable WHERE ROW REGEXP &quot;^\D+&quot; AND VALUE REGEXP ”(?i)hypertable&quot;;

17. Atomic Counters New column option: Modified via existing API using specially formatted values: create table counts ( url COUNTER, ); Reset counter to n =n Decrement counter by n -n Increment counter by n [+]n Description Value Format

18. Group Commit Supports highly concurrent updates Trades minimum latency for better throughput Configurable commit interval per-table: CREATE TABLE counts ( url, domain ) GROUP_COMMIT_INTERVAL=100;

19. Compression Block compression Cell Store (SSTable) blocks Commit Log blocks Supported Compression Schemes: zlib lzo quicklz bmz none

20. Bloom Filter Dramatically reduces disk access Associated with each Cell Store Tells you if key is definitively not present

21. Performance Evaluation

22. Setup Modeled after Test described in Bigtable paper 1 Test Dispatcher, 4 Test Clients, 4 Tablet Servers Test was written entirely in Java Hardware 1 X 1.8 GHz Dual-core Opteron 10 GB RAM 3X 250GB SATA drives Software HDFS 0.20.2 running on all 10 nodes, 3X replication HBase 0.20.4 Hypertable 0.9.3.3

23. Latency

24. Throughput 220 Scan 10 byte values 75 Scan 100 byte values 58 Scan 1KB values 2 Scan 10KB values 129 Sequential Read 100 byte values 68 Sequential Read 1KB values 1060 Sequential Read 10KB values 931 Random Write 10 byte values 427 Random Write 100 byte values 102 Random Write 1KB values 51 Random Write 10KB values 100 Random Read Zipfian 2.5 GB 777 Random Read Zipfian 20 GB 925 Random Read Zipfian 80 GB Hypertable Advantage Relative to HBase (%) Test

25. Why does Performance Matter? $$$

26. Upcoming Release (0.9.5) Last “alpha” release Release Date: February 15th 2011 Features Automatic range balancing Asynchronous API Improved Monitoring System

27. Resources blog .hypertable.com Blog: www.hypertable.org Project Site: hypertable Twitter:

28. Professional Support

29. Q&A