C APIs for Apache HBase by Aditya Kishore

  • 544 views
Uploaded on

Aditya Kishore is a Software Engineer at MapR working on Apache HBase, M7 and Apache Drill. This presentation was given at the HBase MeetUp hosted by Ancestry.com. …

Aditya Kishore is a Software Engineer at MapR working on Apache HBase, M7 and Apache Drill. This presentation was given at the HBase MeetUp hosted by Ancestry.com.

Aditya introduces libHBase, its design goals, code examples, and the future roadmap.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
544
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
1
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. © 2014 MapR Technologies© 2014 MapR Technologies C APIs for Apache HBase Aditya Kishore (aditya@mapr.com) HBase Meetup @ Ancestry.com
  • 2. © 2014 MapR Technologies About Me • Apache HBase contributor since 2012. • Software Engineer at MapR Technologies. • Working on Apache HBase, M7 and Apache Drill. • https://github.com/adityakishore/
  • 3. © 2014 MapR Technologies Agenda • Introducing libHBase • Design goals. • API structures, functions and callbacks. • Client for HBase 0.94.x • Building libHBase and using in your application. • A short example code using the APIs. • Roadmap for the future. • Demo (if time permits).
  • 4. © 2014 MapR Technologies Introducing libHBase • Frequent requests from our customers (and on HBase mailing list). • Long open JIRA HBASE-1015 (created: 21/Nov/08) • “Mandatory for a database approaching 1.0 release” - Nick Dimiduk. • Thrift / REST bindings are the only available solution. • Need for stable, future ready, native APIs. • “C” extern linkage enables support for other languages too. • Work in progress to define standard C APIs and develop the client library. • Join the conversation at HBASE-9835, HBASE-9977, HBASE-10168.
  • 5. © 2014 MapR Technologies • Parity with Java APIs. • With native RPCs to HBase servers without any intermediate gateway. • Uniform APIs across HBase releases to ensure binary compatibility. • Thread-safe, non-blocking, asynchronous client. • Synchronous wrappers available over the asynchronous client. • No support for explicit batching in asynchronous APIs. • Application managed buffers with destructors for library created resources. Design Goals
  • 6. © 2014 MapR Technologies Structures, Functions and Callbacks • The APIs use opaque structures to encapsulate implementation. • Manipulated using create(), set_xyz(), and destroy() methods. • Exception to this rule is hb_cell_t type which has static members. • Asynchronous APIs all take a callback function as parameter. • Invoking an async APIs always return success, error in callback. • Callbacks can be triggered on any thread. • Application are expected to not block in the callback routine.
  • 7. © 2014 MapR Technologies Available now: Client for HBase 0.94.x • Download from MapR’s Github repository http://goo.gl/gs0uMs. • Applied on top of HBase 0.94.17 code base. • Patches available soon on JIRAs and Review Board. • JNI implementation of the proposed APIs using AsyncHBase 1.5. • Requires modified version of AsyncHBase http://goo.gl/aikZDn. • Supports asynchronous Put, Get, Delete and Scan APIs. • Synchronous Admin APIs wraps HBaseAdmin.
  • 8. © 2014 MapR Technologies Building libHBase and using in your Application • Integrated with the HBase maven build using ‘libhbase’ profile. • The HBase native library is built using Gradle. • Header files in ‘/libhbase/include’, link with ‘-lhbase’. • libhbase.so and libjvm.so need to be in LD_LIBRARY_PATH. • Specify JVM classpath using ‘CLASSPATH’ or ‘HBASE_LIB_DIR’. • Other JVM options can be specified using ‘LIBHBASE_OPTS’. • Find the example code under ‘/libhbase/example/async’. • A rudimentary performance test ‘/bin/perftest.sh’ is included.
  • 9. © 2014 MapR Technologies #include <hbase/hbase.h> int main(int argc, char **argv) { hb_connection_t connection = NULL; hb_connection_create("localhost:2181", "/hbase", &connection); hb_client_t client = NULL; hb_client_create(connection, &client); hb_put_t put = NULL; char *row_key = strdup("row_001"); size_t row_key_len = strlen(row_key); hb_put_create((byte_t*) row_key, row_key_len, &put); hb_cell_t *cell = (hb_cell_t*) calloc(1, sizeof(hb_cell_t)); cell->row = (byte_t*)row_key; cell->row_len = row_key_len; {...} // assemble the rest of the cell hb_put_add_cell(put, cell); hb_mutation_send(client, put, mutation_callback, cell); {...} // wait for callback to complete hb_client_destroy(client, NULL, NULL); return 0; } void mutation_callback(int32_t err, hb_client_t client, hb_mutation_t mutation, hb_result_t result, void *extra) { hb_cell_t *cell = (hb_cell_t*) extra; free(cell->row); free(cell); if (result) { hb_result_destroy(result); } hb_mutation_destroy(mutation); {...} // notify waiters } Example Code using the APIs ZK Connection Create the client Prepare the Put Assemble a Cell Add Cell to Put Submit to Client Free the buffers Free the Objects HBase header Free the Client
  • 10. © 2014 MapR Technologies Roadmap for Future • Support for remaining APIs (increment, append, etc...). • Performance test for Get / Scan / Mix workload. • Support for Scan / Get Filters. • Optimization of JNI / native call transitions. • Support for secure RPCs. • Native implementation for HBase 0.96+.
  • 11. © 2014 MapR Technologies© 2014 MapR Technologies DEMO
  • 12. © 2014 MapR Technologies Q & A @mapr maprtech aditya [at] mapr.com Engage with us! MapR maprtech mapr-technologies
  • 13. © 2014 MapR Technologies© 2014 MapR Technologies We are hiring!!! http://www.mapr.com/company/careers