C APIs for Apache HBase by Aditya Kishore


Published on

Aditya Kishore is a Software Engineer at MapR working on Apache HBase, M7 and Apache Drill. This presentation was given at the HBase MeetUp hosted by Ancestry.com.

Aditya introduces libHBase, its design goals, code examples, and the future roadmap.

C APIs for Apache HBase by Aditya Kishore

  1. 1. © 2014 MapR Technologies© 2014 MapR Technologies C APIs for Apache HBase Aditya Kishore (aditya@mapr.com) HBase Meetup @ Ancestry.com
  2. 2. © 2014 MapR Technologies About Me • Apache HBase contributor since 2012. • Software Engineer at MapR Technologies. • Working on Apache HBase, M7 and Apache Drill. • https://github.com/adityakishore/
  3. 3. © 2014 MapR Technologies Agenda • Introducing libHBase • Design goals. • API structures, functions and callbacks. • Client for HBase 0.94.x • Building libHBase and using in your application. • A short example code using the APIs. • Roadmap for the future. • Demo (if time permits).
  4. 4. © 2014 MapR Technologies Introducing libHBase • Frequent requests from our customers (and on HBase mailing list). • Long open JIRA HBASE-1015 (created: 21/Nov/08) • “Mandatory for a database approaching 1.0 release” - Nick Dimiduk. • Thrift / REST bindings are the only available solution. • Need for stable, future ready, native APIs. • “C” extern linkage enables support for other languages too. • Work in progress to define standard C APIs and develop the client library. • Join the conversation at HBASE-9835, HBASE-9977, HBASE-10168.
  5. 5. © 2014 MapR Technologies • Parity with Java APIs. • With native RPCs to HBase servers without any intermediate gateway. • Uniform APIs across HBase releases to ensure binary compatibility. • Thread-safe, non-blocking, asynchronous client. • Synchronous wrappers available over the asynchronous client. • No support for explicit batching in asynchronous APIs. • Application managed buffers with destructors for library created resources. Design Goals
  6. 6. © 2014 MapR Technologies Structures, Functions and Callbacks • The APIs use opaque structures to encapsulate implementation. • Manipulated using create(), set_xyz(), and destroy() methods. • Exception to this rule is hb_cell_t type which has static members. • Asynchronous APIs all take a callback function as parameter. • Invoking an async APIs always return success, error in callback. • Callbacks can be triggered on any thread. • Application are expected to not block in the callback routine.
  7. 7. © 2014 MapR Technologies Available now: Client for HBase 0.94.x • Download from MapR’s Github repository http://goo.gl/gs0uMs. • Applied on top of HBase 0.94.17 code base. • Patches available soon on JIRAs and Review Board. • JNI implementation of the proposed APIs using AsyncHBase 1.5. • Requires modified version of AsyncHBase http://goo.gl/aikZDn. • Supports asynchronous Put, Get, Delete and Scan APIs. • Synchronous Admin APIs wraps HBaseAdmin.
  8. 8. © 2014 MapR Technologies Building libHBase and using in your Application • Integrated with the HBase maven build using ‘libhbase’ profile. • The HBase native library is built using Gradle. • Header files in ‘/libhbase/include’, link with ‘-lhbase’. • libhbase.so and libjvm.so need to be in LD_LIBRARY_PATH. • Specify JVM classpath using ‘CLASSPATH’ or ‘HBASE_LIB_DIR’. • Other JVM options can be specified using ‘LIBHBASE_OPTS’. • Find the example code under ‘/libhbase/example/async’. • A rudimentary performance test ‘/bin/perftest.sh’ is included.
  9. 9. © 2014 MapR Technologies #include <hbase/hbase.h> int main(int argc, char **argv) { hb_connection_t connection = NULL; hb_connection_create("localhost:2181", "/hbase", &connection); hb_client_t client = NULL; hb_client_create(connection, &client); hb_put_t put = NULL; char *row_key = strdup("row_001"); size_t row_key_len = strlen(row_key); hb_put_create((byte_t*) row_key, row_key_len, &put); hb_cell_t *cell = (hb_cell_t*) calloc(1, sizeof(hb_cell_t)); cell->row = (byte_t*)row_key; cell->row_len = row_key_len; {...} // assemble the rest of the cell hb_put_add_cell(put, cell); hb_mutation_send(client, put, mutation_callback, cell); {...} // wait for callback to complete hb_client_destroy(client, NULL, NULL); return 0; } void mutation_callback(int32_t err, hb_client_t client, hb_mutation_t mutation, hb_result_t result, void *extra) { hb_cell_t *cell = (hb_cell_t*) extra; free(cell->row); free(cell); if (result) { hb_result_destroy(result); } hb_mutation_destroy(mutation); {...} // notify waiters } Example Code using the APIs ZK Connection Create the client Prepare the Put Assemble a Cell Add Cell to Put Submit to Client Free the buffers Free the Objects HBase header Free the Client
  10. 10. © 2014 MapR Technologies Roadmap for Future • Support for remaining APIs (increment, append, etc...). • Performance test for Get / Scan / Mix workload. • Support for Scan / Get Filters. • Optimization of JNI / native call transitions. • Support for secure RPCs. • Native implementation for HBase 0.96+.
  11. 11. © 2014 MapR Technologies© 2014 MapR Technologies DEMO
  12. 12. © 2014 MapR Technologies Q & A @mapr maprtech aditya [at] mapr.com Engage with us! MapR maprtech mapr-technologies
  13. 13. © 2014 MapR Technologies© 2014 MapR Technologies We are hiring!!! http://www.mapr.com/company/careers