Downloaded 129 times
More Related Content
![[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...](https://cdn.slidesharecdn.com/ss_thumbnails/dbts-tokyo-sqlite4-20170906-170911071410-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Hypertable Berlin Buzzwords
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Hypertable Berlin Buzzwords
- 1. Hypertable Doug JuddCEO, Hypertable, Inc.
- 2. High Performance, OpenSource Scalable Database Modeled after Bigtable High Performance Implementation (C++) Project Started in March 2007 Thrift Interface for all popular languages Java PHP Ruby Python Perl, etc.
- 3. Bigtable: the infrastructurethat Google is built on YouTube Blogger Google Earth Google Maps Orkut (social network) Gmail Google Analytics Google Book Search Google Code Crawl Database … plus 90 other Google services …
- 4. Functionality Massive sparsetables of information Single primary key index Cells can have mulitple timestamped versions Not Relational No joins (not yet) No secondary indexes (not yet) Not a transaction system (not yet)
- 5.
- 6.
- 7. Auto-Sharding MongoDB AsterDataGreenplum
- 8.
- 9. Dynamo-based Hash TableArchitectures Cassandra Project Voldemort Riak
- 10.
- 11.
- 12. Order Preserving Partitioner(Cassandra) www.recipezaar.com 1091721999…629750272 + www.ribbonprinters.com 1091721999…965293103 / 2 = www.rgb????i?pQdp ?.??? 1091721999…297521687
- 13. Order Preserving PartitionerBalance Problem
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22. Log Structured Merge(LSM) Tree Eliminates random I/O on writes Converts random I/O to sequential I/O Write path Commit log on disk (DFS) In-memory map In-memory map gets “compacted” to disk Disk files periodically get merged
- 23. Range Server Managesranges of table data CellCache: In-memory map containing recent updates CellStore: On-disk (DFS) file containing “compacted” cell cache
- 24. Range Server: CellStoreSequence of 65K blocks of compressed key/value pairs
- 25. Compression Cell Storeblocks are compressed Commit Log updates are compressed Supported Compression Schemes zlib (--best and --fast) lzo quicklz bmz none
- 26. Bloom Filter Probabilisticdata structure associated with every CellStore Indicates if key is not present
- 27. Caching Block CacheCaches CellStore blocks Blocks are cached uncompressed Dynamically adjusted size based on workload Query Cache Caches query results
- 28.
- 29. Performance Evaluation Hypertablevs. HBase
- 30. Test Setup Hypertablev0.9.3.2 (not yet released) HBase 0.20.3 HDFS 0.20.2 10 machines 3 Hyperspace / Zookeeper replicas 1 Master / 4 Tablet Servers (5GB RAM) 1 Test Dispatcher / 4 Test Clients Machine profile 1 X 1.8 GHz Dual-core Opteron 10 GB RAM 3 X 250 GB SATA drives
- 31. Random Write /Sequential Read
- 32.
- 33. Project Resources Twitter: hypertable www.hypertable.org
- 34.
Editor's Notes
- #2 My name is Doug Judd and I'm the CEO of Hypertable, Inc. I'm also the original creator and current maintainer of the project. Today I'm going to present an architectural overview of Hypertable within the context of some of the other popular scalable database designs. I'm also going to present some preliminary results from a performance evaluation that we're in the process of doing, comparing Hypertable with HBase.
- #3 About three years ago, I was working as an architect at Zvents, a local search engine. At the time we were trying to become the "Google" of local search. This meant collecting large and growing amounts of click log and query log data, doing analytics on that data, and using the results to fuel our ranking, recommendation, and ad targetting systems. At the time Hadoop existed, which povided the scalable filesystem and the MapReduce framework, but there was no scalable open source solution for delivering large data sets to live applications. So we decided to do an implementation and Bigtable was the obvious choice. We decided to do it open source and reap all of the benefits that the open source development model has to offer. A big focus of the project has been on optimum performance and the reason is simple. Efficiency gains scale linearly with the system, which translates to reduced hardware to deliver the same capacity. To that end, we chose C++ as the implementation language. Though the system is implemented in C++, we have a Thrift interface that provides language bindings for all popular high-level languages.
- #4 Describe the 360 degree panoramic view feature of Google Maps
- #16 In ‘07, Google was running 100,000 MapReduce jobs and processing 20 petabytes daily
- #18 Describe the 360 degree panoramic view feature of Google Maps
- #22 Describe the 360 degree panoramic view feature of Google Maps