Meetup cassandra sfo_jdbc

  • 1,796 views
Uploaded on

Using Apache Cassandra with CQL via the JDBC Driver, connection pooling and JdbcTemplate from Spring Framework.

Using Apache Cassandra with CQL via the JDBC Driver, connection pooling and JdbcTemplate from Spring Framework.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,796
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
50
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
      How to Integrate Apache Cassandra's CQL with your Existing Java Application
      Nate McCall [email_address] @zznate
  • 2.
      CQL via JDBC
    • (Personal) Motivations for Pushing this Approach:
    • 3. - Make Cassandra useful to wide variety of developers
    • 4. - Easily usable from most common application paradigms
    • 5. * web/ejb containers
    • 6. * service layers
    • 7. - Integrate with existing APIs
    • 8. * Spring Framework's JdbcTemplate in this case
  • 9.
      CQL via JDBC: Status
    • You can start using it, but...
    • 10. - Inflexible typing
    • 11. - Almost no meta data (result or database level)
    • 12. - Other rough edges
    • 13. - Still going over the Thrift API (execute_cql_query)
    • 14. Spend some time knowing what you are getting into
  • 15.
      CQL JDBC Driver: What's Missing
    • Very limited typing support:
    • 16. - Most type specific mutators on Statement interface.
    • 17. Currently supported types:
    • 18. * string
    • 19. * short, int and long
    • 20. * bytes
    • 21. * object
    • 22. * rowId
  • 23.
      CQL JDBC Driver: What's Missing
    • ResultSetMetaData and DatabaseMetadata
    • 24. - Not available in CQL yet
    • 25. - Lot's of tool APIs need are meta data driven
  • 26.
      CQL JDBC Driver: What's Missing
      Audience Participation!
    • Assuming these are the only 3 regex's used to parse statements:
    • 27. ...
    • 28. private static final Pattern Select = Pattern.compile("SELECT...
    • 29. private static final Pattern Update = Pattern.compile("UPDATE...
    • 30. private static final Pattern Delete = Pattern.compile("DELETE...
    • 31.
  • 32.
      CQL JDBC Driver
      Current worst design warts (IMO):
    • - Relies on CF metadata for *all* typing loaded statically at init
    • 33. - Does not respect calls to typed method in Statement hierarchy (easy to fix)
    • 34. - Creates cart-before-horse issue wrt test setup
  • 35.
      CQL JDBC Driver: What Works
    • Driver implementation and setup
    • 36. - Meaningful URL semantics
    • 37. - Completely encapsulates Thrift
    • 38. - Compression (specified from the Thrift API)
    • 39. - Statement parsing (except “insert”!)
  • 40.
      CQL JDBC Driver: What Works
    • Working ResultSet implementation:
    • 41. - CassandraResultSet convenience methods
    • 42. * getColumn/findColumn methods
    • 43. * getKey (but only in bytes!)
  • 44.
      CQL JDBC Driver: What Works
    • Some useful controls on Statement hierarchy:
    • 45. - Allows for fetchDirection and fetchSize on statement
    • 46. - addBatch()
    • 47. - clearParameters()
  • 48.
      CQL JDBC: Making Use of the Driver
      - Need to pool connections to be useful
    • - No existing librariers out there were Cassandra flavored
    • 49. - We knew a little bit about writing Cassandra clients
  • 50.
      CQL JDBC: Cassandra-jdbc-pool (CJP)
    • Lots of functionality specific to Cassandra architecture:
    • 51. - Cluster name, keyspace and at least 1 host required
    • 52. - Additional settings for:
    • 53. * fail over semantics
    • 54. * automatic host discovery
    • 55. * timeout counters and thresholds
  • 56.
      CQL JDBC: CJP Configuration (JNDI)
    • <Resource name= &quot;cassandra/CassandraClientFactory&quot;
    • 57. auth= &quot;Container&quot;
    • 58. type= &quot;me.prettyprint.cassandra.api.Keyspace&quot;
    • 59. factory= &quot;me.prettyprint.cassandra.jndi.CassandraClientJndiResourceFactory&quot;
    • 60. hosts= &quot;cass1:9160,cass2:9160,cass3:9160&quot;
    • 61. user= &quot;user&quot;
    • 62. password= &quot;passwd&quot;
    • 63. keyspace=&quot; Keyspace1&quot;
    • 64. clusterName= &quot;Test Cluster&quot;
    • 65. maxActive= &quot;15&quot;
    • 66. maxWaitTimeWhenExhausted= &quot;10&quot;
    • 67. failoverPolicy= &quot;ON_FAIL_TRY_ALL_AVAILABLE&quot;
    • 68. autoDiscoverHosts= &quot;true&quot;
    • 69. runAutoDiscoveryAtStartup= &quot;true&quot; />
  • 70.
      CQL JDBC: CJP Configuration (Spring)
    • <bean class= &quot;com.datastax.drivers.jdbc.pool.cassandra.jdbc.HCQLDataSource&quot;
    • 71. id= &quot;cqlDataSource&quot; >
    • 72. <property name= &quot;clusterName&quot; value= &quot;TestCluster&quot; />
    • 73. <property name= &quot;keyspaceName&quot; value= &quot;PortfolioDemo&quot; />
    • 74. <property name= &quot;hosts&quot; value= &quot;127.0.0.1:9170&quot; />
    • 75. </bean>
    • 76. <bean class= &quot;org.springframework.jdbc.core.JdbcTemplate&quot;
    • 77. id=&quot; jdbcTemplate&quot; >
    • 78. <constructor-arg ref= &quot;cqlDataSource&quot; />
    • 79. </bean>
  • 80.
      CQL JDBC and Spring Framework
    • JdbcTemplate FTW!
    • 81. - mature: “Since: May 3, 2001”
    • 82. - easy to understand with lots of examples
    • 83. - well documented
    • 84. http://static.springsource.org/spring/docs/3.0.x/javadoc-api/org/springframework/jdbc/core/JdbcTemplate.html
  • 85.
      CQL Spring Framework: JdbcTemplate
      Easiest approach: manual string formatting
    • - can do what you want
    • 86. - BYO encoding
    • 87. - String argument versions of:
    • 88. - update
    • 89. - execute
    • 90. - batchUpdate: array of string statements
    • 91. - queryForInt: count operations
    • 92. - queryForList: multiple rows
    • 93. - queryForMap: single row
  • 94.
      CQL and JdbcTemplate
    • private static final String STOCK_CQL =
    • 95. “ select price FROM Stocks WHERE KEY = ?&quot;;
    • 96. jdbcTemplate.query(STOCK_CQL, stockTicker,
    • 97. new RowMapper<Stock>() {
    • 98. public Stock mapRow(ResultSet rs, int row) throws SQLException {
    • 99. CassandraResultSet crs = (CassandraResultSet)rs;
    • 100. Stock stock = new Stock();
    • 101. stock.setTicker(new String(crs.getKey()));
    • 102. stock.setPrice(crs.getDouble(&quot;price&quot;));
    • 103. return stock;
    • 104. }
    • 105. });
    Reading Data via RowMapper:
  • 106.
      CQL and JdbcTemplate
    • Updates using an object array and positional placeholders
    • 107. private static String UPDATE_PORTOFOLIO_CQL =
    • 108. &quot; update Portfolios set ? = ? where KEY = ? &quot;;
    • 109. jdbcTemplate.update(UPDATE_PORTFOLIO_CQL,
    • 110. new Object[] {position.getTicker(),
    • 111. position.getCount(),
    • 112. portfolio.getName()});
  • 113.
      CQL and JdbcTemplate
    • Batch Update with BatchPreparedStatementSetter
    • 114. private static final String UPDATE_PORT_CQL =
    • 115. &quot;update Portfolios set ? = ? where KEY = ?&quot;;
    • 116. jdbcTemplate.batchUpdate(UPDATE_PORT_CQL,
    • 117. new BatchPreparedStatementSetter() {
    • 118. public void setValues(PreparedStatement ps, int index) throws SQLException {
    • 119. Position pos = portfolio.getConstituents().get(index);
    • 120. ps.setString(1, pos.getTicker());
    • 121. ps.setLong(2, pos.getShares());
    • 122. ps.setString(3,portfolio.getName());
    • 123. }
    • 124. public int getBatchSize() {
    • 125. return portfolio.getConstituents().size();
    • 126. }
    • 127. });
  • 128.
      CQL and JdbcTemplate
    • Deleting with BatchPreparedStatementSetter
    • 129. private static final String DELETE_PORT_COLUMN_CQL =
    • 130. &quot;delete ? From Portfolios where KEY = ?&quot;;
    • 131. jdbcTemplate.batchUpdate(UPDATE_PORT_COLUMN_CQL,
    • 132. new BatchPreparedStatementSetter() {
    • 133. public void setValues(PreparedStatement ps, int index) throws SQLException {
    • 134. Position pos = portfolio.getConstituents().get(index);
    • 135. ps.setString(1, pos.getTicker());
    • 136. ps.setString(2,portfolio.getName());
    • 137. }
    • 138. public int getBatchSize() {
    • 139. return listOfTickersToDelete.size();
    • 140. }
    • 141. });
  • 142.
      CQL and JDBC
    • Summary:
    • 143. - More than a proof of concept
    • 144. - Not quite production ready
    • 145. - Rough edges remain
    • 146. - Ready for experimentation, test drives and bug hunting
    • 147. *** Lot's of folks are now/will always be happy with the existing Thrift API
  • 148.
      Data Model
      GOOG
      AAPL
      NFLX
      NOK
    • price: 589.55
    • price: 401.76
      price: 78.73
      name : Google
      name : Apple
      name : Netflix
      price: 6.90
      name : Nokia
      exchange : NYSE
      Portfolio
  • 149.
      Data Model
      GOOG
      AAPL
      NFLX
      NOK
    • price: 589.55
    • price: 401.76
      price: 78.73
      name : Google
      name : Apple
      name : Netflix
      price: 6.90
      name : Nokia
      exchange : NYSE
      Stocks
  • 150.
      Data Model
      StockHist
      10/25/2011: 6.71
      GOOG
      AAPL
      NFLX
      NOK
      10/24/2011: 6.76
      10/21/2011: 6.61
      10/25/2011: 77.37
      10/24/2011: 118.84
      10/21/2011: 117.04
      10/25/2011: 397.77
      10/24/2011: 405.77
      10/21/2011: 392.87
      10/25/2011: 583.16
      10/24/2011: 596.42
      10/21/2011: 590.49
  • 151.
      CQL via JDBC: Components
    • - HCQLDataSource (from jdbc-pool)
    • 152. - Spring Framework's JdbcTemplate
    • 153. - DAO class with associated domain objects
    • 154. - Junit
    • 155. - Spring Framework's SpringJUnit4ClassRunner (context setup and injection)
    • 156. - EmbededServerHelper from hector-test (manage Cassandra lifecycle, directories and configuration)
  • 157.
      Development Resources
    CQL Documentation (and CQL Shell) http://www.datastax.com/docs/1.0/dml/using_cql Hector Documentation http://hector-client.org
    • Cassandra Maven Plugin (exec-cql goal) http://mojo.codehaus.org/cassandra-maven-plugin/
    • 158. CCM localhost cassandra cluster https://github.com/pcmanus/ccm
    • 159. OpsCenter http://www.datastax.com/products/opscenter
      Cassandra AMIs https://github.com/riptano/CassandraClusterAMI
  • 160.
      Questions
      Nate McCall [email_address] @zznate