Meetup cassandra sfo_jdbc
Upcoming SlideShare
Loading in...5
×
 

Meetup cassandra sfo_jdbc

on

  • 2,323 views

Using Apache Cassandra with CQL via the JDBC Driver, connection pooling and JdbcTemplate from Spring Framework.

Using Apache Cassandra with CQL via the JDBC Driver, connection pooling and JdbcTemplate from Spring Framework.

Statistics

Views

Total Views
2,323
Views on SlideShare
2,323
Embed Views
0

Actions

Likes
0
Downloads
48
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Meetup cassandra sfo_jdbc Meetup cassandra sfo_jdbc Presentation Transcript

      • How to Integrate Apache Cassandra's CQL with your Existing Java Application
        Nate McCall [email_address] @zznate
      • CQL via JDBC
      • (Personal) Motivations for Pushing this Approach:
      • - Make Cassandra useful to wide variety of developers
      • - Easily usable from most common application paradigms
      • * web/ejb containers
      • * service layers
      • - Integrate with existing APIs
      • * Spring Framework's JdbcTemplate in this case
      • CQL via JDBC: Status
      • You can start using it, but...
      • - Inflexible typing
      • - Almost no meta data (result or database level)
      • - Other rough edges
      • - Still going over the Thrift API (execute_cql_query)
      • Spend some time knowing what you are getting into
      • CQL JDBC Driver: What's Missing
      • Very limited typing support:
      • - Most type specific mutators on Statement interface.
      • Currently supported types:
      • * string
      • * short, int and long
      • * bytes
      • * object
      • * rowId
      • CQL JDBC Driver: What's Missing
      • ResultSetMetaData and DatabaseMetadata
      • - Not available in CQL yet
      • - Lot's of tool APIs need are meta data driven
      • CQL JDBC Driver: What's Missing
        Audience Participation!
      • Assuming these are the only 3 regex's used to parse statements:
      • ...
      • private static final Pattern Select = Pattern.compile("SELECT...
      • private static final Pattern Update = Pattern.compile("UPDATE...
      • private static final Pattern Delete = Pattern.compile("DELETE...
      • CQL JDBC Driver
        Current worst design warts (IMO):
      • - Relies on CF metadata for *all* typing loaded statically at init
      • - Does not respect calls to typed method in Statement hierarchy (easy to fix)
      • - Creates cart-before-horse issue wrt test setup
      • CQL JDBC Driver: What Works
      • Driver implementation and setup
      • - Meaningful URL semantics
      • - Completely encapsulates Thrift
      • - Compression (specified from the Thrift API)
      • - Statement parsing (except “insert”!)
      • CQL JDBC Driver: What Works
      • Working ResultSet implementation:
      • - CassandraResultSet convenience methods
      • * getColumn/findColumn methods
      • * getKey (but only in bytes!)
      • CQL JDBC Driver: What Works
      • Some useful controls on Statement hierarchy:
      • - Allows for fetchDirection and fetchSize on statement
      • - addBatch()
      • - clearParameters()
      • CQL JDBC: Making Use of the Driver
        - Need to pool connections to be useful
      • - No existing librariers out there were Cassandra flavored
      • - We knew a little bit about writing Cassandra clients
      • CQL JDBC: Cassandra-jdbc-pool (CJP)
      • Lots of functionality specific to Cassandra architecture:
      • - Cluster name, keyspace and at least 1 host required
      • - Additional settings for:
      • * fail over semantics
      • * automatic host discovery
      • * timeout counters and thresholds
      • CQL JDBC: CJP Configuration (JNDI)
      • <Resource name= &quot;cassandra/CassandraClientFactory&quot;
      • auth= &quot;Container&quot;
      • type= &quot;me.prettyprint.cassandra.api.Keyspace&quot;
      • factory= &quot;me.prettyprint.cassandra.jndi.CassandraClientJndiResourceFactory&quot;
      • hosts= &quot;cass1:9160,cass2:9160,cass3:9160&quot;
      • user= &quot;user&quot;
      • password= &quot;passwd&quot;
      • keyspace=&quot; Keyspace1&quot;
      • clusterName= &quot;Test Cluster&quot;
      • maxActive= &quot;15&quot;
      • maxWaitTimeWhenExhausted= &quot;10&quot;
      • failoverPolicy= &quot;ON_FAIL_TRY_ALL_AVAILABLE&quot;
      • autoDiscoverHosts= &quot;true&quot;
      • runAutoDiscoveryAtStartup= &quot;true&quot; />
      • CQL JDBC: CJP Configuration (Spring)
      • <bean class= &quot;com.datastax.drivers.jdbc.pool.cassandra.jdbc.HCQLDataSource&quot;
      • id= &quot;cqlDataSource&quot; >
      • <property name= &quot;clusterName&quot; value= &quot;TestCluster&quot; />
      • <property name= &quot;keyspaceName&quot; value= &quot;PortfolioDemo&quot; />
      • <property name= &quot;hosts&quot; value= &quot;127.0.0.1:9170&quot; />
      • </bean>
      • <bean class= &quot;org.springframework.jdbc.core.JdbcTemplate&quot;
      • id=&quot; jdbcTemplate&quot; >
      • <constructor-arg ref= &quot;cqlDataSource&quot; />
      • </bean>
      • CQL JDBC and Spring Framework
      • JdbcTemplate FTW!
      • - mature: “Since: May 3, 2001”
      • - easy to understand with lots of examples
      • - well documented
      • http://static.springsource.org/spring/docs/3.0.x/javadoc-api/org/springframework/jdbc/core/JdbcTemplate.html
      • CQL Spring Framework: JdbcTemplate
        Easiest approach: manual string formatting
      • - can do what you want
      • - BYO encoding
      • - String argument versions of:
      • - update
      • - execute
      • - batchUpdate: array of string statements
      • - queryForInt: count operations
      • - queryForList: multiple rows
      • - queryForMap: single row
      • CQL and JdbcTemplate
      • private static final String STOCK_CQL =
      • “ select price FROM Stocks WHERE KEY = ?&quot;;
      • jdbcTemplate.query(STOCK_CQL, stockTicker,
      • new RowMapper<Stock>() {
      • public Stock mapRow(ResultSet rs, int row) throws SQLException {
      • CassandraResultSet crs = (CassandraResultSet)rs;
      • Stock stock = new Stock();
      • stock.setTicker(new String(crs.getKey()));
      • stock.setPrice(crs.getDouble(&quot;price&quot;));
      • return stock;
      • }
      • });
      Reading Data via RowMapper:
      • CQL and JdbcTemplate
      • Updates using an object array and positional placeholders
      • private static String UPDATE_PORTOFOLIO_CQL =
      • &quot; update Portfolios set ? = ? where KEY = ? &quot;;
      • jdbcTemplate.update(UPDATE_PORTFOLIO_CQL,
      • new Object[] {position.getTicker(),
      • position.getCount(),
      • portfolio.getName()});
      • CQL and JdbcTemplate
      • Batch Update with BatchPreparedStatementSetter
      • private static final String UPDATE_PORT_CQL =
      • &quot;update Portfolios set ? = ? where KEY = ?&quot;;
      • jdbcTemplate.batchUpdate(UPDATE_PORT_CQL,
      • new BatchPreparedStatementSetter() {
      • public void setValues(PreparedStatement ps, int index) throws SQLException {
      • Position pos = portfolio.getConstituents().get(index);
      • ps.setString(1, pos.getTicker());
      • ps.setLong(2, pos.getShares());
      • ps.setString(3,portfolio.getName());
      • }
      • public int getBatchSize() {
      • return portfolio.getConstituents().size();
      • }
      • });
      • CQL and JdbcTemplate
      • Deleting with BatchPreparedStatementSetter
      • private static final String DELETE_PORT_COLUMN_CQL =
      • &quot;delete ? From Portfolios where KEY = ?&quot;;
      • jdbcTemplate.batchUpdate(UPDATE_PORT_COLUMN_CQL,
      • new BatchPreparedStatementSetter() {
      • public void setValues(PreparedStatement ps, int index) throws SQLException {
      • Position pos = portfolio.getConstituents().get(index);
      • ps.setString(1, pos.getTicker());
      • ps.setString(2,portfolio.getName());
      • }
      • public int getBatchSize() {
      • return listOfTickersToDelete.size();
      • }
      • });
      • CQL and JDBC
      • Summary:
      • - More than a proof of concept
      • - Not quite production ready
      • - Rough edges remain
      • - Ready for experimentation, test drives and bug hunting
      • *** Lot's of folks are now/will always be happy with the existing Thrift API
      • Data Model
        GOOG
        AAPL
        NFLX
        NOK
      • price: 589.55
      • price: 401.76
        price: 78.73
        name : Google
        name : Apple
        name : Netflix
        price: 6.90
        name : Nokia
        exchange : NYSE
        Portfolio
      • Data Model
        GOOG
        AAPL
        NFLX
        NOK
      • price: 589.55
      • price: 401.76
        price: 78.73
        name : Google
        name : Apple
        name : Netflix
        price: 6.90
        name : Nokia
        exchange : NYSE
        Stocks
      • Data Model
        StockHist
        10/25/2011: 6.71
        GOOG
        AAPL
        NFLX
        NOK
        10/24/2011: 6.76
        10/21/2011: 6.61
        10/25/2011: 77.37
        10/24/2011: 118.84
        10/21/2011: 117.04
        10/25/2011: 397.77
        10/24/2011: 405.77
        10/21/2011: 392.87
        10/25/2011: 583.16
        10/24/2011: 596.42
        10/21/2011: 590.49
      • CQL via JDBC: Components
      • - HCQLDataSource (from jdbc-pool)
      • - Spring Framework's JdbcTemplate
      • - DAO class with associated domain objects
      • - Junit
      • - Spring Framework's SpringJUnit4ClassRunner (context setup and injection)
      • - EmbededServerHelper from hector-test (manage Cassandra lifecycle, directories and configuration)
      • Development Resources
      CQL Documentation (and CQL Shell) http://www.datastax.com/docs/1.0/dml/using_cql Hector Documentation http://hector-client.org
      • Cassandra Maven Plugin (exec-cql goal) http://mojo.codehaus.org/cassandra-maven-plugin/
      • CCM localhost cassandra cluster https://github.com/pcmanus/ccm
      • OpsCenter http://www.datastax.com/products/opscenter
        Cassandra AMIs https://github.com/riptano/CassandraClusterAMI
      • Questions
        Nate McCall [email_address] @zznate