SlideShare a Scribd company logo
1 of 51
HBase Coprocessors
  Deploy shared functionality
    directly on the cluster

        O’Reilly Webcast
       November 4th, 2011
About Me
• Solutions Architect @ Cloudera
• Apache HBase & Whirr Committer
• Author of
      HBase – The Definitive Guide
• Working with HBase since end
  of 2007
• Organizer of the Munich OpenHUG
• Speaker at Conferences (Fosdem, Hadoop World)
Overview
• Coprocessors were added to Bigtable
  – Mentioned during LADIS 2009 talk
• Runs user code within each region of a table
  – Code split and moves with region
• Defines high level call interface for clients
• Calls addressed to rows or ranges of rows
• Implicit automatic scaling, load balancing, and
  request routing
Examples Use-Cases
• Bigtable uses Coprocessors
  –   Scalable metadata management
  –   Distributed language model for machine translation
  –   Distributed query processing for full-text index
  –   Regular expression search in code repository
• MapReduce jobs over HBase are often map-only
  jobs
  – Row keys are already sorted and distinct
  ➜ Could be replaced by Coprocessors
HBase Coprocessors
• Inspired by Google’s Coprocessors
   – Not much information available, but general idea is
     understood
• Define various types of server-side code extensions
   –   Associated with table using a table property
   –   Attribute is a path to JAR file
   –   JAR is loaded when region is opened
   –   Blends new functionality with existing
• Can be chained with Priorities and Load Order

➜ Allows for dynamic RPC extensions
Coprocessor Classes and Interfaces
• The Coprocessor Interface
  – All user code must inherit from this class
• The CoprocessorEnvironment Interface
  – Retains state across invocations
  – Predefined classes
• The CoprocessorHost Interface
  – Ties state and user code together
  – Predefined classes
Coprocessor Priority
• System or User

/** Highest installation priority */
static final int PRIORITY_HIGHEST = 0;
/** High (system) installation priority */
static final int PRIORITY_SYSTEM = Integer.MAX_VALUE / 4;
/** Default installation prio for user coprocessors */
static final int PRIORITY_USER = Integer.MAX_VALUE / 2;
/** Lowest installation priority */
static final int PRIORITY_LOWEST = Integer.MAX_VALUE;
Coprocessor Environment
• Available Methods
Coprocessor Host
• Maintains all Coprocessor instances and their
  environments (state)
• Concrete Classes
  – MasterCoprocessorHost
  – RegionCoprocessorHost
  – WALCoprocessorHost
• Subclasses provide access to specialized
  Environment implementations
Control Flow
Coprocessor Interface
• Base for all other types of Coprocessors
• start() and stop() methods for lifecycle
  management
• State as defined in the interface:
Observer Classes
• Comparable to database triggers
  – Callback functions/hooks for every explicit API
    method, but also all important internal calls
• Concrete Implementations
  – MasterObserver
     • Hooks into HMaster API
  – RegionObserver
     • Hooks into Region related operations
  – WALObserver
     • Hooks into write-ahead log operations
Region Observers
• Can mediate (veto) actions
  – Used by the security policy extensions
  – Priority allows mediators to run first
• Hooks into all CRUD+S API calls and more
  – get(), put(), delete(), scan(), increment(),…
  – checkAndPut(), checkAndDelete(),…
  – flush(), compact(), split(),…
• Pre/Post Hooks for every call
• Can be used to build secondary indexes, filters
Endpoint Classes
• Define a dynamic RPC protocol, used between
  client and region server
• Executes arbitrary code, loaded in region server
  – Future development will add code weaving/inspection
    to deny any malicious code
• Steps to add your own methods
  – Define and implement your own protocol
  – Implement endpoint coprocessor
  – Call HTable’s coprocessorExec() or coprocessorProxy()
Coprocessor Loading
• There are two ways: dynamic or static
   – Static: use configuration files and table schema
   – Dynamic: not available (yet)
• For static loading from configuration:
   – Order is important (defines the execution order)
   – Special property key for each host type
   – Region related classes are loaded for all regions and
     tables
   – Priority is always System
   – JAR must be on class path
Loading from Configuration
• Example:
  <property>
   <name>hbase.coprocessor.region.classes</name>
   <value>coprocessor.RegionObserverExample, 
    coprocessor.AnotherCoprocessor</value>
  </property>
  <property>
   <name>hbase.coprocessor.master.classes</name>
   <value>coprocessor.MasterObserverExample</value>
  </property>
  <property>
   <name>hbase.coprocessor.wal.classes</name>
   <value>coprocessor.WALObserverExample, 
    bar.foo.MyWALObserver</value>
  </property>
Coprocessor Loading (cont.)
• For static loading from table schema:
  – Definition per table
  – For all regions of the table
  – Only region related classes, not WAL or Master
  – Added to HTableDescriptor, when table is created
    or altered
  – Allows to set the priority and JAR path
  COPROCESSOR$<num>
   <path-to-jar>|<classname>|<priority>
Loading from Table Schema
• Example:
'COPROCESSOR$1' => 
 'hdfs://localhost:8020/users/leon/test.jar| 
  coprocessor.Test|10'

'COPROCESSOR$2' => 
 '/Users/laura/test2.jar| 
  coprocessor.AnotherTest|1000'
Example: Add Coprocessor
public static void main(String[] args) throws IOException {
  Configuration conf = HBaseConfiguration.create();
  FileSystem fs = FileSystem.get(conf);
  Path path = new Path(fs.getUri() + Path.SEPARATOR +
   "test.jar");
  HTableDescriptor htd = new HTableDescriptor("testtable");
  htd.addFamily(new HColumnDescriptor("colfam1"));
  htd.setValue("COPROCESSOR$1", path.toString() +
   "|" + RegionObserverExample.class.getCanonicalName() +
   "|" + Coprocessor.PRIORITY_USER);
  HBaseAdmin admin = new HBaseAdmin(conf);
  admin.createTable(htd);
  System.out.println(admin.getTableDescriptor(
   Bytes.toBytes("testtable")));
}
Example Output
{NAME => 'testtable', COPROCESSOR$1 =>
'file:/test.jar|coprocessor.RegionObserverExample|1073741823',
FAMILIES => [{NAME => 'colfam1', BLOOMFILTER => 'NONE',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE',
VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536',
IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}
Region Observers
• Handles all region related events
• Hooks for two classes of operations:
  – Lifecycle changes
  – Client API Calls
• All client API calls have a pre/post hook
  – Can be used to grant access on preGet()
  – Can be used to update secondary indexes on
    postPut()
Handling Region Lifecycle Events



• Hook into pending open, open, and pending close
  state changes
• Called implicitly by the framework
   – preOpen(), postOpen(),…
• Used to piggyback or fail the process, e.g.
   – Cache warm up after a region opens
   – Suppress region splitting, compactions, flushes
Region Environment
Special Hook Parameter
public interface RegionObserver extends Coprocessor {

 /**
  * Called before the region is reported as open to the master.
  * @param c the environment provided by the region server
  */
 void preOpen(final
   ObserverContext<RegionCoprocessorEnvironment> c);

 /**
  * Called after the region is reported as open to the master.
  * @param c the environment provided by the region server
  */
 void postOpen(final
   ObserverContext<RegionCoprocessorEnvironment> c);
ObserverContext
Chain of Command
• Especially the complete() and bypass()
  methods allow to change the processing chain
  – complete() ends the chain at the current
    coprocessor
  – bypass() completes the pre/post chain but uses
    the last value returned by the coprocessors,
    possibly not calling the actual API method (for
    pre-hooks)
Example: Pre-Hook Complete


@Override
public void preSplit(ObserverContext
     <RegionCoprocessorEnvironment> e) {
  e.complete();
}
Master Observer
• Handles all HMaster related events
  – DDL type calls, e.g. create table, add column
  – Region management calls, e.g. move, assign
• Pre/post hooks with Context
• Specialized environment provided
Master Environment
Master Services (cont.)
• Very powerful features
  – Access the AssignmentManager to modify plans
  – Access the MasterFileSystem to create or access
    resources on HDFS
  – Access the ServerManager to get the list of known
    servers
  – Use the ExecutorService to run system-wide
    background processes
• Be careful (for now)!
Example: Master Post Hook
public class MasterObserverExample
  extends BaseMasterObserver {
  @Override public void postCreateTable(
    ObserverContext<MasterCoprocessorEnvironment> env,
    HRegionInfo[] regions, boolean sync)
    throws IOException {
    String tableName =
      regions[0].getTableDesc().getNameAsString();
    MasterServices services =
      env.getEnvironment().getMasterServices();
    MasterFileSystem masterFileSystem =
     services.getMasterFileSystem();
    FileSystem fileSystem = masterFileSystem.getFileSystem();
    Path blobPath = new Path(tableName + "-blobs");
    fileSystem.mkdirs(blobPath);
  }
}
Example Output
hbase(main):001:0> create 'testtable',
  'colfam1‘
0 row(s) in 0.4300 seconds

$ bin/hadoop dfs -ls
  Found 1 items
  drwxr-xr-x - larsgeorge supergroup 0 ...
  /user/larsgeorge/testtable-blobs
Endpoints
• Dynamic RPC extends server-side functionality
  – Useful for MapReduce like implementations
  – Handles the Map part server-side, Reduce needs
    to be done client side
• Based on CoprocessorProtocol interface
• Routing to regions is based on either single
  row keys, or row key ranges
  – Call is sent, no matter if row exists or not since
    region start and end keys are coarse grained
Custom Endpoint Implementation
• Involves two steps:
  – Extend the CoprocessorProtocol interface
     • Defines the actual protocol
  – Extend the BaseEndpointCoprocessor
     • Provides the server-side code and the dynamic RPC
       method
Example: Row Count Protocol
public interface RowCountProtocol
  extends CoprocessorProtocol {
  long getRowCount()
    throws IOException;
  long getRowCount(Filter filter)
    throws IOException;
  long getKeyValueCount()
    throws IOException;
}
Example: Endpoint for Row Count
public class RowCountEndpoint
extends BaseEndpointCoprocessor
implements RowCountProtocol {

 private long getCount(Filter filter,
  boolean countKeyValues) throws IOException {
   Scan scan = new Scan();
  scan.setMaxVersions(1);
  if (filter != null) {
    scan.setFilter(filter);
  }
Example: Endpoint for Row Count
RegionCoprocessorEnvironment environment =
  (RegionCoprocessorEnvironment)
  getEnvironment();
// use an internal scanner to perform
// scanning.
InternalScanner scanner =
  environment.getRegion().getScanner(scan);
int result = 0;
Example: Endpoint for Row Count
    try {
      List<KeyValue> curVals =
        new ArrayList<KeyValue>();
      boolean done = false;
      do {
        curVals.clear();
        done = scanner.next(curVals);
        result += countKeyValues ? curVals.size() : 1;
      } while (done);
    } finally {
      scanner.close();
    }
    return result;
}
Example: Endpoint for Row Count
    @Override
    public long getRowCount() throws IOException {
      return getRowCount(new FirstKeyOnlyFilter());
    }

    @Override
    public long getRowCount(Filter filter) throws IOException {
     return getCount(filter, false);
    }

    @Override
    public long getKeyValueCount() throws IOException {
      return getCount(null, true);
    }
}
Endpoint Invocation
• There are two ways to invoke the call
  – By Proxy, using HTable.coprocessorProxy()
     • Uses a delayed model, i.e. the call is send when the proxied
       method is invoked
  – By Exec, using HTable.coprocessorExec()
     • The call is send in parallel to all regions and the results are
       collected immediately
• The Batch.Call class is used be coprocessorExec()
  to wrap the calls per region
• The optional Batch.Callback can be used to react
  upon completion of the remote call
Exec vs. Proxy
Example: Invocation by Exec
public static void main(String[] args) throws IOException {
 Configuration conf = HBaseConfiguration.create();
 HTable table = new HTable(conf, "testtable");
 try {
   Map<byte[], Long> results =
    table.coprocessorExec(RowCountProtocol.class, null, null,
    new Batch.Call<RowCountProtocol, Long>() {
      @Override
      public Long call(RowCountProtocol counter)
      throws IOException {
        return counter.getRowCount();
      }
    });
Example: Invocation by Exec
      long total = 0;
      for (Map.Entry<byte[], Long> entry :
           results.entrySet()) {
        total += entry.getValue().longValue();
        System.out.println("Region: " +
          Bytes.toString(entry.getKey()) +
          ", Count: " + entry.getValue());
      }
      System.out.println("Total Count: " + total);
    } catch (Throwable throwable) {
       throwable.printStackTrace();
    }
}
Example Output
Region:
  testtable,,1303417572005.51f9e2251c...cb
  cb0c66858f., Count: 2
Region: testtable,row3,
  1303417572005.7f3df4dcba...dbc99fce5d
  87., Count: 3
Total Count: 5
Batch Convenience
• The Batch.forMethod() helps to quickly map a
  protocol function into a Batch.Call
• Useful for single method calls to the servers
• Uses the Java reflection API to retrieve the
  named method
• Saves you from implementing the anonymous
  inline class
Batch Convenience
Batch.Call call =
 Batch.forMethod(
   RowCountProtocol.class,
   "getKeyValueCount");
Map<byte[], Long> results =
 table.coprocessorExec(
   RowCountProtocol.class,
   null, null, call);
Call Multiple Endpoints
• Sometimes you need to call more than one
  endpoint in a single roundtrip call to the
  servers
• This requires an anonymous inline class, since
  Batch.forMethod cannot handle this
Call Multiple Endpoints
Map<byte[], Pair<Long, Long>>
results = table.coprocessorExec(
 RowCountProtocol.class, null, null,
 new Batch.Call<RowCountProtocol,
   Pair<Long, Long>>() {
   public Pair<Long, Long> call(
     RowCountProtocol counter)
   throws IOException {
      return new Pair(
     counter.getRowCount(),
     counter.getKeyValueCount());
   }
 });
Example: Invocation by Proxy
RowCountProtocol protocol =
  table.coprocessorProxy(
    RowCountProtocol.class,
    Bytes.toBytes("row4"));
long rowsInRegion =
  protocol.getRowCount();
  System.out.println(
    "Region Row Count: " +
    rowsInRegion);
Questions?

• Contact:
  Email: lars@cloudera.com
  Twitter: @larsgeorge

• Talk at Hadoop World, November 8th & 9th
Special Offer for
                                        Webcast Attendees
                                    Visit http://oreilly.com to
                                    purchase your copy of
                                    Hbase: The Definitive
                                    Guide and enter code
                                    4CAST to save 40% off
                                    print book & 50% off
                                    ebook with special code
                                    4CAST


Visit http://oreilly.com/webcasts to view upcoming webcasts and online events.

More Related Content

What's hot

HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...
HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...
HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...Michael Stack
 
PostgreSQL 8.4 TriLUG 2009-11-12
PostgreSQL 8.4 TriLUG 2009-11-12PostgreSQL 8.4 TriLUG 2009-11-12
PostgreSQL 8.4 TriLUG 2009-11-12Andrew Dunstan
 
Oracle ACFS High Availability NFS Services (HANFS)
Oracle ACFS High Availability NFS Services (HANFS)Oracle ACFS High Availability NFS Services (HANFS)
Oracle ACFS High Availability NFS Services (HANFS)Anju Garg
 
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, Adjust
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, AdjustShipping Data from Postgres to Clickhouse, by Murat Kabilov, Adjust
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, AdjustAltinity Ltd
 
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...Flink Forward
 
Apache Con NA 2013 - Cassandra Internals
Apache Con NA 2013 - Cassandra InternalsApache Con NA 2013 - Cassandra Internals
Apache Con NA 2013 - Cassandra Internalsaaronmorton
 
Let's Compare: A Benchmark review of InfluxDB and Elasticsearch
Let's Compare: A Benchmark review of InfluxDB and ElasticsearchLet's Compare: A Benchmark review of InfluxDB and Elasticsearch
Let's Compare: A Benchmark review of InfluxDB and ElasticsearchInfluxData
 
Extending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session ExtensionsExtending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session ExtensionsDatabricks
 
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...InfluxData
 
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxData
 
January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing
January 2015 HUG: Apache Flink:  Fast and reliable large-scale data processingJanuary 2015 HUG: Apache Flink:  Fast and reliable large-scale data processing
January 2015 HUG: Apache Flink: Fast and reliable large-scale data processingYahoo Developer Network
 
ACID Transactions in Hive
ACID Transactions in HiveACID Transactions in Hive
ACID Transactions in HiveEugene Koifman
 
CRX2Oak - all the secrets of repository migration
CRX2Oak - all the secrets of repository migrationCRX2Oak - all the secrets of repository migration
CRX2Oak - all the secrets of repository migrationTomasz Rękawek
 
Introduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerIntroduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerMydbops
 
Streaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit LogStreaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit LogJoe Stein
 
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 Taipei
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 TaipeiPostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 Taipei
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 TaipeiSatoshi Nagayasu
 
Sql Server 2008 New Programmability Features
Sql Server 2008 New Programmability FeaturesSql Server 2008 New Programmability Features
Sql Server 2008 New Programmability Featuressqlserver.co.il
 

What's hot (20)

HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...
HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...
HBaseConEast2016: How yarn timeline service v.2 unlocks 360 degree platform i...
 
PostgreSQL 8.4 TriLUG 2009-11-12
PostgreSQL 8.4 TriLUG 2009-11-12PostgreSQL 8.4 TriLUG 2009-11-12
PostgreSQL 8.4 TriLUG 2009-11-12
 
Oracle ACFS High Availability NFS Services (HANFS)
Oracle ACFS High Availability NFS Services (HANFS)Oracle ACFS High Availability NFS Services (HANFS)
Oracle ACFS High Availability NFS Services (HANFS)
 
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, Adjust
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, AdjustShipping Data from Postgres to Clickhouse, by Murat Kabilov, Adjust
Shipping Data from Postgres to Clickhouse, by Murat Kabilov, Adjust
 
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...
Flink Forward Berlin 2017: Stefan Richter - A look at Flink's internal data s...
 
Apache Con NA 2013 - Cassandra Internals
Apache Con NA 2013 - Cassandra InternalsApache Con NA 2013 - Cassandra Internals
Apache Con NA 2013 - Cassandra Internals
 
Let's Compare: A Benchmark review of InfluxDB and Elasticsearch
Let's Compare: A Benchmark review of InfluxDB and ElasticsearchLet's Compare: A Benchmark review of InfluxDB and Elasticsearch
Let's Compare: A Benchmark review of InfluxDB and Elasticsearch
 
Perl Programming - 04 Programming Database
Perl Programming - 04 Programming DatabasePerl Programming - 04 Programming Database
Perl Programming - 04 Programming Database
 
Extending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session ExtensionsExtending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session Extensions
 
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...
Mixing Metrics and Logs with Grafana + Influx by David Kaltschmidt, Director ...
 
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
 
January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing
January 2015 HUG: Apache Flink:  Fast and reliable large-scale data processingJanuary 2015 HUG: Apache Flink:  Fast and reliable large-scale data processing
January 2015 HUG: Apache Flink: Fast and reliable large-scale data processing
 
ACID Transactions in Hive
ACID Transactions in HiveACID Transactions in Hive
ACID Transactions in Hive
 
Hadoop on osx
Hadoop on osxHadoop on osx
Hadoop on osx
 
CRX2Oak - all the secrets of repository migration
CRX2Oak - all the secrets of repository migrationCRX2Oak - all the secrets of repository migration
CRX2Oak - all the secrets of repository migration
 
Introduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizerIntroduction to Mongodb execution plan and optimizer
Introduction to Mongodb execution plan and optimizer
 
Streaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit LogStreaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit Log
 
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 Taipei
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 TaipeiPostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 Taipei
PostgreSQL 9.4, 9.5 and Beyond @ COSCUP 2015 Taipei
 
Sql Server 2008 New Programmability Features
Sql Server 2008 New Programmability FeaturesSql Server 2008 New Programmability Features
Sql Server 2008 New Programmability Features
 
Presto overview
Presto overviewPresto overview
Presto overview
 

Viewers also liked

U.S. Senate Social Graph, 1991 - Present
U.S. Senate Social Graph, 1991 - PresentU.S. Senate Social Graph, 1991 - Present
U.S. Senate Social Graph, 1991 - PresentO'Reilly Media
 
InsideRIA Outlook for 2009
InsideRIA Outlook for 2009InsideRIA Outlook for 2009
InsideRIA Outlook for 2009AndreCharland
 
Oct. 14, 2011 webcast ch7 subnets bruce hartpence
Oct. 14, 2011 webcast ch7 subnets bruce hartpenceOct. 14, 2011 webcast ch7 subnets bruce hartpence
Oct. 14, 2011 webcast ch7 subnets bruce hartpenceO'Reilly Media
 
2 3-2012 Take Control of iCloud
2 3-2012 Take Control of iCloud2 3-2012 Take Control of iCloud
2 3-2012 Take Control of iCloudO'Reilly Media
 
12 13 what is desktop virtualization
12 13 what is desktop virtualization12 13 what is desktop virtualization
12 13 what is desktop virtualizationO'Reilly Media
 
Sxsw speaker submission_effectiveui_07252014
Sxsw speaker submission_effectiveui_07252014Sxsw speaker submission_effectiveui_07252014
Sxsw speaker submission_effectiveui_07252014patrickVinson
 
Test Driven Development
Test Driven DevelopmentTest Driven Development
Test Driven DevelopmentJohn Blanco
 
WattzOn Personal Energy Audit
WattzOn Personal Energy AuditWattzOn Personal Energy Audit
WattzOn Personal Energy AuditWeb 2.0 Expo
 
Citizen Science on the Move conference 25, 26 & 27 june 2012
Citizen Science on the Move conference 25, 26 & 27 june 2012Citizen Science on the Move conference 25, 26 & 27 june 2012
Citizen Science on the Move conference 25, 26 & 27 june 2012Ronald Lenz
 
Souders WPO Web 2.0 Expo
Souders WPO Web 2.0 ExpoSouders WPO Web 2.0 Expo
Souders WPO Web 2.0 ExpoSteve Souders
 
Search Different Understanding Apple's New Search Engine State of Search 2016
Search Different   Understanding Apple's New Search Engine State of Search 2016Search Different   Understanding Apple's New Search Engine State of Search 2016
Search Different Understanding Apple's New Search Engine State of Search 2016Andrew Shotland
 
Sharing Apache's Goodness: How We Should be Telling Apache's Story
Sharing Apache's Goodness: How We Should be Telling Apache's StorySharing Apache's Goodness: How We Should be Telling Apache's Story
Sharing Apache's Goodness: How We Should be Telling Apache's StoryJoe Brockmeier
 
Open Source at the Apache Software Foundation
Open Source at the Apache Software Foundation Open Source at the Apache Software Foundation
Open Source at the Apache Software Foundation wgstoddard
 
But we're already open source! Why would I want to bring my code to Apache?
But we're already open source! Why would I want to bring my code to Apache?But we're already open source! Why would I want to bring my code to Apache?
But we're already open source! Why would I want to bring my code to Apache?gagravarr
 
Voice+IP Conference Frankfurt, Germany
Voice+IP Conference Frankfurt, GermanyVoice+IP Conference Frankfurt, Germany
Voice+IP Conference Frankfurt, GermanyMarc René Gardeya
 
Apple earnings q4-2010
Apple earnings q4-2010Apple earnings q4-2010
Apple earnings q4-2010O'Reilly Media
 

Viewers also liked (20)

U.S. Senate Social Graph, 1991 - Present
U.S. Senate Social Graph, 1991 - PresentU.S. Senate Social Graph, 1991 - Present
U.S. Senate Social Graph, 1991 - Present
 
InsideRIA Outlook for 2009
InsideRIA Outlook for 2009InsideRIA Outlook for 2009
InsideRIA Outlook for 2009
 
Oct. 14, 2011 webcast ch7 subnets bruce hartpence
Oct. 14, 2011 webcast ch7 subnets bruce hartpenceOct. 14, 2011 webcast ch7 subnets bruce hartpence
Oct. 14, 2011 webcast ch7 subnets bruce hartpence
 
2 3-2012 Take Control of iCloud
2 3-2012 Take Control of iCloud2 3-2012 Take Control of iCloud
2 3-2012 Take Control of iCloud
 
12 13 what is desktop virtualization
12 13 what is desktop virtualization12 13 what is desktop virtualization
12 13 what is desktop virtualization
 
Sxsw speaker submission_effectiveui_07252014
Sxsw speaker submission_effectiveui_07252014Sxsw speaker submission_effectiveui_07252014
Sxsw speaker submission_effectiveui_07252014
 
Hoppala at ARE2011
Hoppala at ARE2011Hoppala at ARE2011
Hoppala at ARE2011
 
Test Driven Development
Test Driven DevelopmentTest Driven Development
Test Driven Development
 
WattzOn Personal Energy Audit
WattzOn Personal Energy AuditWattzOn Personal Energy Audit
WattzOn Personal Energy Audit
 
Hoppala at XMediaLab
Hoppala at XMediaLabHoppala at XMediaLab
Hoppala at XMediaLab
 
Citizen Science on the Move conference 25, 26 & 27 june 2012
Citizen Science on the Move conference 25, 26 & 27 june 2012Citizen Science on the Move conference 25, 26 & 27 june 2012
Citizen Science on the Move conference 25, 26 & 27 june 2012
 
Souders WPO Web 2.0 Expo
Souders WPO Web 2.0 ExpoSouders WPO Web 2.0 Expo
Souders WPO Web 2.0 Expo
 
Search Different Understanding Apple's New Search Engine State of Search 2016
Search Different   Understanding Apple's New Search Engine State of Search 2016Search Different   Understanding Apple's New Search Engine State of Search 2016
Search Different Understanding Apple's New Search Engine State of Search 2016
 
Sharing Apache's Goodness: How We Should be Telling Apache's Story
Sharing Apache's Goodness: How We Should be Telling Apache's StorySharing Apache's Goodness: How We Should be Telling Apache's Story
Sharing Apache's Goodness: How We Should be Telling Apache's Story
 
Open Source at the Apache Software Foundation
Open Source at the Apache Software Foundation Open Source at the Apache Software Foundation
Open Source at the Apache Software Foundation
 
But we're already open source! Why would I want to bring my code to Apache?
But we're already open source! Why would I want to bring my code to Apache?But we're already open source! Why would I want to bring my code to Apache?
But we're already open source! Why would I want to bring my code to Apache?
 
Allister Frost Speaker Biography
Allister Frost Speaker BiographyAllister Frost Speaker Biography
Allister Frost Speaker Biography
 
Voice+IP Conference Frankfurt, Germany
Voice+IP Conference Frankfurt, GermanyVoice+IP Conference Frankfurt, Germany
Voice+IP Conference Frankfurt, Germany
 
2009 Research Where
2009 Research Where2009 Research Where
2009 Research Where
 
Apple earnings q4-2010
Apple earnings q4-2010Apple earnings q4-2010
Apple earnings q4-2010
 

Similar to HBase Coprocessors: Deploy Shared Functionality Directly on the Cluster

HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...Cloudera, Inc.
 
H base introduction & development
H base introduction & developmentH base introduction & development
H base introduction & developmentShashwat Shriparv
 
Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117exsuns
 
Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014Michael Renner
 
nodejs_at_a_glance.ppt
nodejs_at_a_glance.pptnodejs_at_a_glance.ppt
nodejs_at_a_glance.pptWalaSidhom1
 
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...BIOVIA
 
Airflow tutorials hands_on
Airflow tutorials hands_onAirflow tutorials hands_on
Airflow tutorials hands_onpko89403
 
HBase Coprocessors @ HUG NYC
HBase Coprocessors @ HUG NYCHBase Coprocessors @ HUG NYC
HBase Coprocessors @ HUG NYCmlai
 
Copper: A high performance workflow engine
Copper: A high performance workflow engineCopper: A high performance workflow engine
Copper: A high performance workflow enginedmoebius
 
Introduction to LAVA Workload Scheduler
Introduction to LAVA Workload SchedulerIntroduction to LAVA Workload Scheduler
Introduction to LAVA Workload SchedulerNopparat Nopkuat
 
Infrastructure modeling with chef
Infrastructure modeling with chefInfrastructure modeling with chef
Infrastructure modeling with chefCharles Johnson
 
Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Lucas Jellema
 
Java colombo-deep-dive-into-jax-rs
Java colombo-deep-dive-into-jax-rsJava colombo-deep-dive-into-jax-rs
Java colombo-deep-dive-into-jax-rsSagara Gunathunga
 
Getting to know Laravel 5
Getting to know Laravel 5Getting to know Laravel 5
Getting to know Laravel 5Bukhori Aqid
 
Fortify aws aurora_proxy_2019_pleu
Fortify aws aurora_proxy_2019_pleuFortify aws aurora_proxy_2019_pleu
Fortify aws aurora_proxy_2019_pleuMarco Tusa
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache MesosJoe Stein
 
The Evolution of a Relational Database Layer over HBase
The Evolution of a Relational Database Layer over HBaseThe Evolution of a Relational Database Layer over HBase
The Evolution of a Relational Database Layer over HBaseDataWorks Summit
 

Similar to HBase Coprocessors: Deploy Shared Functionality Directly on the Cluster (20)

HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...
HBaseCon 2012 | HBase Coprocessors – Deploy Shared Functionality Directly on ...
 
H base introduction & development
H base introduction & developmentH base introduction & development
H base introduction & development
 
Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117
 
Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014
 
nodejs_at_a_glance.ppt
nodejs_at_a_glance.pptnodejs_at_a_glance.ppt
nodejs_at_a_glance.ppt
 
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
 
Airflow tutorials hands_on
Airflow tutorials hands_onAirflow tutorials hands_on
Airflow tutorials hands_on
 
HBase Coprocessors @ HUG NYC
HBase Coprocessors @ HUG NYCHBase Coprocessors @ HUG NYC
HBase Coprocessors @ HUG NYC
 
Copper: A high performance workflow engine
Copper: A high performance workflow engineCopper: A high performance workflow engine
Copper: A high performance workflow engine
 
Introduction to LAVA Workload Scheduler
Introduction to LAVA Workload SchedulerIntroduction to LAVA Workload Scheduler
Introduction to LAVA Workload Scheduler
 
Infrastructure modeling with chef
Infrastructure modeling with chefInfrastructure modeling with chef
Infrastructure modeling with chef
 
Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)
 
Java colombo-deep-dive-into-jax-rs
Java colombo-deep-dive-into-jax-rsJava colombo-deep-dive-into-jax-rs
Java colombo-deep-dive-into-jax-rs
 
API
APIAPI
API
 
Get to know PostgreSQL!
Get to know PostgreSQL!Get to know PostgreSQL!
Get to know PostgreSQL!
 
Hadoop
HadoopHadoop
Hadoop
 
Getting to know Laravel 5
Getting to know Laravel 5Getting to know Laravel 5
Getting to know Laravel 5
 
Fortify aws aurora_proxy_2019_pleu
Fortify aws aurora_proxy_2019_pleuFortify aws aurora_proxy_2019_pleu
Fortify aws aurora_proxy_2019_pleu
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache Mesos
 
The Evolution of a Relational Database Layer over HBase
The Evolution of a Relational Database Layer over HBaseThe Evolution of a Relational Database Layer over HBase
The Evolution of a Relational Database Layer over HBase
 

More from O'Reilly Media

2 7-2012 Google how links boost rankings
2 7-2012 Google how links boost rankings2 7-2012 Google how links boost rankings
2 7-2012 Google how links boost rankingsO'Reilly Media
 
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+February 8, 2012 Webcast: 10 Things You Didn't Know About Google+
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+O'Reilly Media
 
Sept. 28, 2011 webcast become an expert google searcher in an hour stephan ...
Sept. 28, 2011 webcast become an expert google searcher in an hour   stephan ...Sept. 28, 2011 webcast become an expert google searcher in an hour   stephan ...
Sept. 28, 2011 webcast become an expert google searcher in an hour stephan ...O'Reilly Media
 
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...O'Reilly Media
 
Oct. 27, 2011 webcast practical and pragmatic application of pmi standards
Oct. 27, 2011 webcast practical and pragmatic application of pmi standardsOct. 27, 2011 webcast practical and pragmatic application of pmi standards
Oct. 27, 2011 webcast practical and pragmatic application of pmi standardsO'Reilly Media
 
Nov. 8, 2011 webcast desiging mobile interfaces by steven hoober
Nov. 8, 2011 webcast   desiging mobile interfaces by steven hooberNov. 8, 2011 webcast   desiging mobile interfaces by steven hoober
Nov. 8, 2011 webcast desiging mobile interfaces by steven hooberO'Reilly Media
 
Oct. 25. 2011 webcast conduct aninterview
Oct. 25. 2011 webcast   conduct aninterviewOct. 25. 2011 webcast   conduct aninterview
Oct. 25. 2011 webcast conduct aninterviewO'Reilly Media
 
Nov. 15, 2011 dani nordin talking to clients about drupal projects
Nov. 15, 2011 dani nordin talking to clients about drupal projectsNov. 15, 2011 dani nordin talking to clients about drupal projects
Nov. 15, 2011 dani nordin talking to clients about drupal projectsO'Reilly Media
 
What's New & Cool in Drupal 7
What's New & Cool in Drupal 7What's New & Cool in Drupal 7
What's New & Cool in Drupal 7O'Reilly Media
 
Dealing with Legacy Perl Code - Peter Scott
Dealing with Legacy Perl Code - Peter ScottDealing with Legacy Perl Code - Peter Scott
Dealing with Legacy Perl Code - Peter ScottO'Reilly Media
 
The Science of Social Media
The Science of Social MediaThe Science of Social Media
The Science of Social MediaO'Reilly Media
 
Web 2.0 Expo Ny--How to Submit a Winning Proposal
Web 2.0 Expo Ny--How to Submit a Winning ProposalWeb 2.0 Expo Ny--How to Submit a Winning Proposal
Web 2.0 Expo Ny--How to Submit a Winning ProposalO'Reilly Media
 
O'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Media
 
Active Facebook Users By Country & Region: August 2009
Active Facebook Users By Country & Region: August 2009Active Facebook Users By Country & Region: August 2009
Active Facebook Users By Country & Region: August 2009O'Reilly Media
 
Twitter Webcast Power Tips, Pt 2
Twitter Webcast Power Tips, Pt 2Twitter Webcast Power Tips, Pt 2
Twitter Webcast Power Tips, Pt 2O'Reilly Media
 
Twitter Webcast Power Tips, Pt 1
Twitter Webcast Power Tips, Pt 1Twitter Webcast Power Tips, Pt 1
Twitter Webcast Power Tips, Pt 1O'Reilly Media
 
Facebook and Myspace App Platforms: A Brief Update
Facebook and Myspace App Platforms: A Brief UpdateFacebook and Myspace App Platforms: A Brief Update
Facebook and Myspace App Platforms: A Brief UpdateO'Reilly Media
 
U.S. iTunes App Store: Sellers
U.S. iTunes App Store: SellersU.S. iTunes App Store: Sellers
U.S. iTunes App Store: SellersO'Reilly Media
 
The What Why And Who Of Xbrl
The What Why And Who Of XbrlThe What Why And Who Of Xbrl
The What Why And Who Of XbrlO'Reilly Media
 

More from O'Reilly Media (20)

2 7-2012 Google how links boost rankings
2 7-2012 Google how links boost rankings2 7-2012 Google how links boost rankings
2 7-2012 Google how links boost rankings
 
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+February 8, 2012 Webcast: 10 Things You Didn't Know About Google+
February 8, 2012 Webcast: 10 Things You Didn't Know About Google+
 
Sept. 28, 2011 webcast become an expert google searcher in an hour stephan ...
Sept. 28, 2011 webcast become an expert google searcher in an hour   stephan ...Sept. 28, 2011 webcast become an expert google searcher in an hour   stephan ...
Sept. 28, 2011 webcast become an expert google searcher in an hour stephan ...
 
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...
Oct. 4, 2011 webcast top 5 tips for building viral social web applications an...
 
Oct. 27, 2011 webcast practical and pragmatic application of pmi standards
Oct. 27, 2011 webcast practical and pragmatic application of pmi standardsOct. 27, 2011 webcast practical and pragmatic application of pmi standards
Oct. 27, 2011 webcast practical and pragmatic application of pmi standards
 
Nov. 8, 2011 webcast desiging mobile interfaces by steven hoober
Nov. 8, 2011 webcast   desiging mobile interfaces by steven hooberNov. 8, 2011 webcast   desiging mobile interfaces by steven hoober
Nov. 8, 2011 webcast desiging mobile interfaces by steven hoober
 
Oct. 25. 2011 webcast conduct aninterview
Oct. 25. 2011 webcast   conduct aninterviewOct. 25. 2011 webcast   conduct aninterview
Oct. 25. 2011 webcast conduct aninterview
 
Nov. 15, 2011 dani nordin talking to clients about drupal projects
Nov. 15, 2011 dani nordin talking to clients about drupal projectsNov. 15, 2011 dani nordin talking to clients about drupal projects
Nov. 15, 2011 dani nordin talking to clients about drupal projects
 
What's New & Cool in Drupal 7
What's New & Cool in Drupal 7What's New & Cool in Drupal 7
What's New & Cool in Drupal 7
 
Dealing with Legacy Perl Code - Peter Scott
Dealing with Legacy Perl Code - Peter ScottDealing with Legacy Perl Code - Peter Scott
Dealing with Legacy Perl Code - Peter Scott
 
The Science of Social Media
The Science of Social MediaThe Science of Social Media
The Science of Social Media
 
Web 2.0 Expo Ny--How to Submit a Winning Proposal
Web 2.0 Expo Ny--How to Submit a Winning ProposalWeb 2.0 Expo Ny--How to Submit a Winning Proposal
Web 2.0 Expo Ny--How to Submit a Winning Proposal
 
O'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Webcast: Architecting Applications For The Cloud
O'Reilly Webcast: Architecting Applications For The Cloud
 
Active Facebook Users By Country & Region: August 2009
Active Facebook Users By Country & Region: August 2009Active Facebook Users By Country & Region: August 2009
Active Facebook Users By Country & Region: August 2009
 
Web Squared
Web SquaredWeb Squared
Web Squared
 
Twitter Webcast Power Tips, Pt 2
Twitter Webcast Power Tips, Pt 2Twitter Webcast Power Tips, Pt 2
Twitter Webcast Power Tips, Pt 2
 
Twitter Webcast Power Tips, Pt 1
Twitter Webcast Power Tips, Pt 1Twitter Webcast Power Tips, Pt 1
Twitter Webcast Power Tips, Pt 1
 
Facebook and Myspace App Platforms: A Brief Update
Facebook and Myspace App Platforms: A Brief UpdateFacebook and Myspace App Platforms: A Brief Update
Facebook and Myspace App Platforms: A Brief Update
 
U.S. iTunes App Store: Sellers
U.S. iTunes App Store: SellersU.S. iTunes App Store: Sellers
U.S. iTunes App Store: Sellers
 
The What Why And Who Of Xbrl
The What Why And Who Of XbrlThe What Why And Who Of Xbrl
The What Why And Who Of Xbrl
 

Recently uploaded

What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 

HBase Coprocessors: Deploy Shared Functionality Directly on the Cluster

  • 1. HBase Coprocessors Deploy shared functionality directly on the cluster O’Reilly Webcast November 4th, 2011
  • 2. About Me • Solutions Architect @ Cloudera • Apache HBase & Whirr Committer • Author of HBase – The Definitive Guide • Working with HBase since end of 2007 • Organizer of the Munich OpenHUG • Speaker at Conferences (Fosdem, Hadoop World)
  • 3. Overview • Coprocessors were added to Bigtable – Mentioned during LADIS 2009 talk • Runs user code within each region of a table – Code split and moves with region • Defines high level call interface for clients • Calls addressed to rows or ranges of rows • Implicit automatic scaling, load balancing, and request routing
  • 4. Examples Use-Cases • Bigtable uses Coprocessors – Scalable metadata management – Distributed language model for machine translation – Distributed query processing for full-text index – Regular expression search in code repository • MapReduce jobs over HBase are often map-only jobs – Row keys are already sorted and distinct ➜ Could be replaced by Coprocessors
  • 5. HBase Coprocessors • Inspired by Google’s Coprocessors – Not much information available, but general idea is understood • Define various types of server-side code extensions – Associated with table using a table property – Attribute is a path to JAR file – JAR is loaded when region is opened – Blends new functionality with existing • Can be chained with Priorities and Load Order ➜ Allows for dynamic RPC extensions
  • 6. Coprocessor Classes and Interfaces • The Coprocessor Interface – All user code must inherit from this class • The CoprocessorEnvironment Interface – Retains state across invocations – Predefined classes • The CoprocessorHost Interface – Ties state and user code together – Predefined classes
  • 7. Coprocessor Priority • System or User /** Highest installation priority */ static final int PRIORITY_HIGHEST = 0; /** High (system) installation priority */ static final int PRIORITY_SYSTEM = Integer.MAX_VALUE / 4; /** Default installation prio for user coprocessors */ static final int PRIORITY_USER = Integer.MAX_VALUE / 2; /** Lowest installation priority */ static final int PRIORITY_LOWEST = Integer.MAX_VALUE;
  • 9. Coprocessor Host • Maintains all Coprocessor instances and their environments (state) • Concrete Classes – MasterCoprocessorHost – RegionCoprocessorHost – WALCoprocessorHost • Subclasses provide access to specialized Environment implementations
  • 11. Coprocessor Interface • Base for all other types of Coprocessors • start() and stop() methods for lifecycle management • State as defined in the interface:
  • 12. Observer Classes • Comparable to database triggers – Callback functions/hooks for every explicit API method, but also all important internal calls • Concrete Implementations – MasterObserver • Hooks into HMaster API – RegionObserver • Hooks into Region related operations – WALObserver • Hooks into write-ahead log operations
  • 13. Region Observers • Can mediate (veto) actions – Used by the security policy extensions – Priority allows mediators to run first • Hooks into all CRUD+S API calls and more – get(), put(), delete(), scan(), increment(),… – checkAndPut(), checkAndDelete(),… – flush(), compact(), split(),… • Pre/Post Hooks for every call • Can be used to build secondary indexes, filters
  • 14. Endpoint Classes • Define a dynamic RPC protocol, used between client and region server • Executes arbitrary code, loaded in region server – Future development will add code weaving/inspection to deny any malicious code • Steps to add your own methods – Define and implement your own protocol – Implement endpoint coprocessor – Call HTable’s coprocessorExec() or coprocessorProxy()
  • 15. Coprocessor Loading • There are two ways: dynamic or static – Static: use configuration files and table schema – Dynamic: not available (yet) • For static loading from configuration: – Order is important (defines the execution order) – Special property key for each host type – Region related classes are loaded for all regions and tables – Priority is always System – JAR must be on class path
  • 16. Loading from Configuration • Example: <property> <name>hbase.coprocessor.region.classes</name> <value>coprocessor.RegionObserverExample, coprocessor.AnotherCoprocessor</value> </property> <property> <name>hbase.coprocessor.master.classes</name> <value>coprocessor.MasterObserverExample</value> </property> <property> <name>hbase.coprocessor.wal.classes</name> <value>coprocessor.WALObserverExample, bar.foo.MyWALObserver</value> </property>
  • 17. Coprocessor Loading (cont.) • For static loading from table schema: – Definition per table – For all regions of the table – Only region related classes, not WAL or Master – Added to HTableDescriptor, when table is created or altered – Allows to set the priority and JAR path COPROCESSOR$<num> <path-to-jar>|<classname>|<priority>
  • 18. Loading from Table Schema • Example: 'COPROCESSOR$1' => 'hdfs://localhost:8020/users/leon/test.jar| coprocessor.Test|10' 'COPROCESSOR$2' => '/Users/laura/test2.jar| coprocessor.AnotherTest|1000'
  • 19. Example: Add Coprocessor public static void main(String[] args) throws IOException { Configuration conf = HBaseConfiguration.create(); FileSystem fs = FileSystem.get(conf); Path path = new Path(fs.getUri() + Path.SEPARATOR + "test.jar"); HTableDescriptor htd = new HTableDescriptor("testtable"); htd.addFamily(new HColumnDescriptor("colfam1")); htd.setValue("COPROCESSOR$1", path.toString() + "|" + RegionObserverExample.class.getCanonicalName() + "|" + Coprocessor.PRIORITY_USER); HBaseAdmin admin = new HBaseAdmin(conf); admin.createTable(htd); System.out.println(admin.getTableDescriptor( Bytes.toBytes("testtable"))); }
  • 20. Example Output {NAME => 'testtable', COPROCESSOR$1 => 'file:/test.jar|coprocessor.RegionObserverExample|1073741823', FAMILIES => [{NAME => 'colfam1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}
  • 21. Region Observers • Handles all region related events • Hooks for two classes of operations: – Lifecycle changes – Client API Calls • All client API calls have a pre/post hook – Can be used to grant access on preGet() – Can be used to update secondary indexes on postPut()
  • 22. Handling Region Lifecycle Events • Hook into pending open, open, and pending close state changes • Called implicitly by the framework – preOpen(), postOpen(),… • Used to piggyback or fail the process, e.g. – Cache warm up after a region opens – Suppress region splitting, compactions, flushes
  • 24. Special Hook Parameter public interface RegionObserver extends Coprocessor { /** * Called before the region is reported as open to the master. * @param c the environment provided by the region server */ void preOpen(final ObserverContext<RegionCoprocessorEnvironment> c); /** * Called after the region is reported as open to the master. * @param c the environment provided by the region server */ void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c);
  • 26. Chain of Command • Especially the complete() and bypass() methods allow to change the processing chain – complete() ends the chain at the current coprocessor – bypass() completes the pre/post chain but uses the last value returned by the coprocessors, possibly not calling the actual API method (for pre-hooks)
  • 27. Example: Pre-Hook Complete @Override public void preSplit(ObserverContext <RegionCoprocessorEnvironment> e) { e.complete(); }
  • 28. Master Observer • Handles all HMaster related events – DDL type calls, e.g. create table, add column – Region management calls, e.g. move, assign • Pre/post hooks with Context • Specialized environment provided
  • 30. Master Services (cont.) • Very powerful features – Access the AssignmentManager to modify plans – Access the MasterFileSystem to create or access resources on HDFS – Access the ServerManager to get the list of known servers – Use the ExecutorService to run system-wide background processes • Be careful (for now)!
  • 31. Example: Master Post Hook public class MasterObserverExample extends BaseMasterObserver { @Override public void postCreateTable( ObserverContext<MasterCoprocessorEnvironment> env, HRegionInfo[] regions, boolean sync) throws IOException { String tableName = regions[0].getTableDesc().getNameAsString(); MasterServices services = env.getEnvironment().getMasterServices(); MasterFileSystem masterFileSystem = services.getMasterFileSystem(); FileSystem fileSystem = masterFileSystem.getFileSystem(); Path blobPath = new Path(tableName + "-blobs"); fileSystem.mkdirs(blobPath); } }
  • 32. Example Output hbase(main):001:0> create 'testtable', 'colfam1‘ 0 row(s) in 0.4300 seconds $ bin/hadoop dfs -ls Found 1 items drwxr-xr-x - larsgeorge supergroup 0 ... /user/larsgeorge/testtable-blobs
  • 33. Endpoints • Dynamic RPC extends server-side functionality – Useful for MapReduce like implementations – Handles the Map part server-side, Reduce needs to be done client side • Based on CoprocessorProtocol interface • Routing to regions is based on either single row keys, or row key ranges – Call is sent, no matter if row exists or not since region start and end keys are coarse grained
  • 34. Custom Endpoint Implementation • Involves two steps: – Extend the CoprocessorProtocol interface • Defines the actual protocol – Extend the BaseEndpointCoprocessor • Provides the server-side code and the dynamic RPC method
  • 35. Example: Row Count Protocol public interface RowCountProtocol extends CoprocessorProtocol { long getRowCount() throws IOException; long getRowCount(Filter filter) throws IOException; long getKeyValueCount() throws IOException; }
  • 36. Example: Endpoint for Row Count public class RowCountEndpoint extends BaseEndpointCoprocessor implements RowCountProtocol { private long getCount(Filter filter, boolean countKeyValues) throws IOException { Scan scan = new Scan(); scan.setMaxVersions(1); if (filter != null) { scan.setFilter(filter); }
  • 37. Example: Endpoint for Row Count RegionCoprocessorEnvironment environment = (RegionCoprocessorEnvironment) getEnvironment(); // use an internal scanner to perform // scanning. InternalScanner scanner = environment.getRegion().getScanner(scan); int result = 0;
  • 38. Example: Endpoint for Row Count try { List<KeyValue> curVals = new ArrayList<KeyValue>(); boolean done = false; do { curVals.clear(); done = scanner.next(curVals); result += countKeyValues ? curVals.size() : 1; } while (done); } finally { scanner.close(); } return result; }
  • 39. Example: Endpoint for Row Count @Override public long getRowCount() throws IOException { return getRowCount(new FirstKeyOnlyFilter()); } @Override public long getRowCount(Filter filter) throws IOException { return getCount(filter, false); } @Override public long getKeyValueCount() throws IOException { return getCount(null, true); } }
  • 40. Endpoint Invocation • There are two ways to invoke the call – By Proxy, using HTable.coprocessorProxy() • Uses a delayed model, i.e. the call is send when the proxied method is invoked – By Exec, using HTable.coprocessorExec() • The call is send in parallel to all regions and the results are collected immediately • The Batch.Call class is used be coprocessorExec() to wrap the calls per region • The optional Batch.Callback can be used to react upon completion of the remote call
  • 42. Example: Invocation by Exec public static void main(String[] args) throws IOException { Configuration conf = HBaseConfiguration.create(); HTable table = new HTable(conf, "testtable"); try { Map<byte[], Long> results = table.coprocessorExec(RowCountProtocol.class, null, null, new Batch.Call<RowCountProtocol, Long>() { @Override public Long call(RowCountProtocol counter) throws IOException { return counter.getRowCount(); } });
  • 43. Example: Invocation by Exec long total = 0; for (Map.Entry<byte[], Long> entry : results.entrySet()) { total += entry.getValue().longValue(); System.out.println("Region: " + Bytes.toString(entry.getKey()) + ", Count: " + entry.getValue()); } System.out.println("Total Count: " + total); } catch (Throwable throwable) { throwable.printStackTrace(); } }
  • 44. Example Output Region: testtable,,1303417572005.51f9e2251c...cb cb0c66858f., Count: 2 Region: testtable,row3, 1303417572005.7f3df4dcba...dbc99fce5d 87., Count: 3 Total Count: 5
  • 45. Batch Convenience • The Batch.forMethod() helps to quickly map a protocol function into a Batch.Call • Useful for single method calls to the servers • Uses the Java reflection API to retrieve the named method • Saves you from implementing the anonymous inline class
  • 46. Batch Convenience Batch.Call call = Batch.forMethod( RowCountProtocol.class, "getKeyValueCount"); Map<byte[], Long> results = table.coprocessorExec( RowCountProtocol.class, null, null, call);
  • 47. Call Multiple Endpoints • Sometimes you need to call more than one endpoint in a single roundtrip call to the servers • This requires an anonymous inline class, since Batch.forMethod cannot handle this
  • 48. Call Multiple Endpoints Map<byte[], Pair<Long, Long>> results = table.coprocessorExec( RowCountProtocol.class, null, null, new Batch.Call<RowCountProtocol, Pair<Long, Long>>() { public Pair<Long, Long> call( RowCountProtocol counter) throws IOException { return new Pair( counter.getRowCount(), counter.getKeyValueCount()); } });
  • 49. Example: Invocation by Proxy RowCountProtocol protocol = table.coprocessorProxy( RowCountProtocol.class, Bytes.toBytes("row4")); long rowsInRegion = protocol.getRowCount(); System.out.println( "Region Row Count: " + rowsInRegion);
  • 50. Questions? • Contact: Email: lars@cloudera.com Twitter: @larsgeorge • Talk at Hadoop World, November 8th & 9th
  • 51. Special Offer for Webcast Attendees Visit http://oreilly.com to purchase your copy of Hbase: The Definitive Guide and enter code 4CAST to save 40% off print book & 50% off ebook with special code 4CAST Visit http://oreilly.com/webcasts to view upcoming webcasts and online events.