<ul>How to Integrate Apache Cassandra's CQL with your Existing Java Application </ul><ul>Nate McCall [email_address] @zzna...
<ul>CQL via JDBC </ul><ul><li>(Personal) Motivations for Pushing this Approach:
- Make Cassandra useful to wide variety of developers
- Easily usable from most common application paradigms
* web/ejb containers
* service layers
- Integrate with existing APIs
* Spring Framework's JdbcTemplate in this case </li></ul>
<ul>CQL via JDBC: Status </ul><ul><li>You can start using it, but...
- Inflexible typing
- Almost no meta data (result or database level)
- Other rough edges
- Still going over the Thrift API (execute_cql_query)
Spend some time knowing what you are getting into </li></ul>
<ul>CQL JDBC Driver: What's Missing </ul><ul><li>Very limited typing support:
- Most type specific mutators on Statement interface.
Currently supported types:
* string
* short, int and long
* bytes
* object
* rowId </li></ul>
<ul>CQL JDBC Driver: What's Missing </ul><ul><li>ResultSetMetaData and DatabaseMetadata
- Not available in CQL yet
- Lot's of tool APIs need are meta data driven </li></ul>
<ul>CQL JDBC Driver: What's Missing </ul><ul>Audience Participation! <li>Assuming these are the only 3 regex's used to par...
...
Upcoming SlideShare
Loading in...5
×

Meetup cassandra sfo_jdbc

1,914

Published on

Using Apache Cassandra with CQL via the JDBC Driver, connection pooling and JdbcTemplate from Spring Framework.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,914
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
53
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Meetup cassandra sfo_jdbc

  1. 1. <ul>How to Integrate Apache Cassandra's CQL with your Existing Java Application </ul><ul>Nate McCall [email_address] @zznate </ul>
  2. 2. <ul>CQL via JDBC </ul><ul><li>(Personal) Motivations for Pushing this Approach:
  3. 3. - Make Cassandra useful to wide variety of developers
  4. 4. - Easily usable from most common application paradigms
  5. 5. * web/ejb containers
  6. 6. * service layers
  7. 7. - Integrate with existing APIs
  8. 8. * Spring Framework's JdbcTemplate in this case </li></ul>
  9. 9. <ul>CQL via JDBC: Status </ul><ul><li>You can start using it, but...
  10. 10. - Inflexible typing
  11. 11. - Almost no meta data (result or database level)
  12. 12. - Other rough edges
  13. 13. - Still going over the Thrift API (execute_cql_query)
  14. 14. Spend some time knowing what you are getting into </li></ul>
  15. 15. <ul>CQL JDBC Driver: What's Missing </ul><ul><li>Very limited typing support:
  16. 16. - Most type specific mutators on Statement interface.
  17. 17. Currently supported types:
  18. 18. * string
  19. 19. * short, int and long
  20. 20. * bytes
  21. 21. * object
  22. 22. * rowId </li></ul>
  23. 23. <ul>CQL JDBC Driver: What's Missing </ul><ul><li>ResultSetMetaData and DatabaseMetadata
  24. 24. - Not available in CQL yet
  25. 25. - Lot's of tool APIs need are meta data driven </li></ul>
  26. 26. <ul>CQL JDBC Driver: What's Missing </ul><ul>Audience Participation! <li>Assuming these are the only 3 regex's used to parse statements:
  27. 27. ...
  28. 28. private static final Pattern Select = Pattern.compile(&quot;SELECT...
  29. 29. private static final Pattern Update = Pattern.compile(&quot;UPDATE...
  30. 30. private static final Pattern Delete = Pattern.compile(&quot;DELETE...
  31. 31. … </li></ul>
  32. 32. <ul>CQL JDBC Driver </ul><ul>Current worst design warts (IMO): <li>- Relies on CF metadata for *all* typing loaded statically at init
  33. 33. - Does not respect calls to typed method in Statement hierarchy (easy to fix)
  34. 34. - Creates cart-before-horse issue wrt test setup </li></ul>
  35. 35. <ul>CQL JDBC Driver: What Works </ul><ul><li>Driver implementation and setup
  36. 36. - Meaningful URL semantics
  37. 37. - Completely encapsulates Thrift
  38. 38. - Compression (specified from the Thrift API)
  39. 39. - Statement parsing (except “insert”!) </li></ul>
  40. 40. <ul>CQL JDBC Driver: What Works </ul><ul><li>Working ResultSet implementation:
  41. 41. - CassandraResultSet convenience methods
  42. 42. * getColumn/findColumn methods
  43. 43. * getKey (but only in bytes!) </li></ul>
  44. 44. <ul>CQL JDBC Driver: What Works </ul><ul><li>Some useful controls on Statement hierarchy:
  45. 45. - Allows for fetchDirection and fetchSize on statement
  46. 46. - addBatch()
  47. 47. - clearParameters() </li></ul>
  48. 48. <ul>CQL JDBC: Making Use of the Driver </ul><ul>- Need to pool connections to be useful <li>- No existing librariers out there were Cassandra flavored
  49. 49. - We knew a little bit about writing Cassandra clients </li></ul>
  50. 50. <ul>CQL JDBC: Cassandra-jdbc-pool (CJP) </ul><ul><li>Lots of functionality specific to Cassandra architecture:
  51. 51. - Cluster name, keyspace and at least 1 host required
  52. 52. - Additional settings for:
  53. 53. * fail over semantics
  54. 54. * automatic host discovery
  55. 55. * timeout counters and thresholds </li></ul>
  56. 56. <ul>CQL JDBC: CJP Configuration (JNDI) </ul><ul><li><Resource name= &quot;cassandra/CassandraClientFactory&quot;
  57. 57. auth= &quot;Container&quot;
  58. 58. type= &quot;me.prettyprint.cassandra.api.Keyspace&quot;
  59. 59. factory= &quot;me.prettyprint.cassandra.jndi.CassandraClientJndiResourceFactory&quot;
  60. 60. hosts= &quot;cass1:9160,cass2:9160,cass3:9160&quot;
  61. 61. user= &quot;user&quot;
  62. 62. password= &quot;passwd&quot;
  63. 63. keyspace=&quot; Keyspace1&quot;
  64. 64. clusterName= &quot;Test Cluster&quot;
  65. 65. maxActive= &quot;15&quot;
  66. 66. maxWaitTimeWhenExhausted= &quot;10&quot;
  67. 67. failoverPolicy= &quot;ON_FAIL_TRY_ALL_AVAILABLE&quot;
  68. 68. autoDiscoverHosts= &quot;true&quot;
  69. 69. runAutoDiscoveryAtStartup= &quot;true&quot; /> </li></ul>
  70. 70. <ul>CQL JDBC: CJP Configuration (Spring) </ul><ul><li><bean class= &quot;com.datastax.drivers.jdbc.pool.cassandra.jdbc.HCQLDataSource&quot;
  71. 71. id= &quot;cqlDataSource&quot; >
  72. 72. <property name= &quot;clusterName&quot; value= &quot;TestCluster&quot; />
  73. 73. <property name= &quot;keyspaceName&quot; value= &quot;PortfolioDemo&quot; />
  74. 74. <property name= &quot;hosts&quot; value= &quot;127.0.0.1:9170&quot; />
  75. 75. </bean>
  76. 76. <bean class= &quot;org.springframework.jdbc.core.JdbcTemplate&quot;
  77. 77. id=&quot; jdbcTemplate&quot; >
  78. 78. <constructor-arg ref= &quot;cqlDataSource&quot; />
  79. 79. </bean> </li></ul>
  80. 80. <ul>CQL JDBC and Spring Framework </ul><ul><li>JdbcTemplate FTW!
  81. 81. - mature: “Since: May 3, 2001”
  82. 82. - easy to understand with lots of examples
  83. 83. - well documented
  84. 84. http://static.springsource.org/spring/docs/3.0.x/javadoc-api/org/springframework/jdbc/core/JdbcTemplate.html </li></ul>
  85. 85. <ul>CQL Spring Framework: JdbcTemplate </ul><ul>Easiest approach: manual string formatting <li>- can do what you want
  86. 86. - BYO encoding
  87. 87. - String argument versions of:
  88. 88. - update
  89. 89. - execute
  90. 90. - batchUpdate: array of string statements
  91. 91. - queryForInt: count operations
  92. 92. - queryForList: multiple rows
  93. 93. - queryForMap: single row </li></ul>
  94. 94. <ul>CQL and JdbcTemplate </ul><ul><li>private static final String STOCK_CQL =
  95. 95. “ select price FROM Stocks WHERE KEY = ?&quot;;
  96. 96. jdbcTemplate.query(STOCK_CQL, stockTicker,
  97. 97. new RowMapper<Stock>() {
  98. 98. public Stock mapRow(ResultSet rs, int row) throws SQLException {
  99. 99. CassandraResultSet crs = (CassandraResultSet)rs;
  100. 100. Stock stock = new Stock();
  101. 101. stock.setTicker(new String(crs.getKey()));
  102. 102. stock.setPrice(crs.getDouble(&quot;price&quot;));
  103. 103. return stock;
  104. 104. }
  105. 105. }); </li></ul>Reading Data via RowMapper:
  106. 106. <ul>CQL and JdbcTemplate </ul><ul><li>Updates using an object array and positional placeholders
  107. 107. private static String UPDATE_PORTOFOLIO_CQL =
  108. 108. &quot; update Portfolios set ? = ? where KEY = ? &quot;;
  109. 109. jdbcTemplate.update(UPDATE_PORTFOLIO_CQL,
  110. 110. new Object[] {position.getTicker(),
  111. 111. position.getCount(),
  112. 112. portfolio.getName()}); </li></ul>
  113. 113. <ul>CQL and JdbcTemplate </ul><ul><li>Batch Update with BatchPreparedStatementSetter
  114. 114. private static final String UPDATE_PORT_CQL =
  115. 115. &quot;update Portfolios set ? = ? where KEY = ?&quot;;
  116. 116. jdbcTemplate.batchUpdate(UPDATE_PORT_CQL,
  117. 117. new BatchPreparedStatementSetter() {
  118. 118. public void setValues(PreparedStatement ps, int index) throws SQLException {
  119. 119. Position pos = portfolio.getConstituents().get(index);
  120. 120. ps.setString(1, pos.getTicker());
  121. 121. ps.setLong(2, pos.getShares());
  122. 122. ps.setString(3,portfolio.getName());
  123. 123. }
  124. 124. public int getBatchSize() {
  125. 125. return portfolio.getConstituents().size();
  126. 126. }
  127. 127. }); </li></ul>
  128. 128. <ul>CQL and JdbcTemplate </ul><ul><li>Deleting with BatchPreparedStatementSetter
  129. 129. private static final String DELETE_PORT_COLUMN_CQL =
  130. 130. &quot;delete ? From Portfolios where KEY = ?&quot;;
  131. 131. jdbcTemplate.batchUpdate(UPDATE_PORT_COLUMN_CQL,
  132. 132. new BatchPreparedStatementSetter() {
  133. 133. public void setValues(PreparedStatement ps, int index) throws SQLException {
  134. 134. Position pos = portfolio.getConstituents().get(index);
  135. 135. ps.setString(1, pos.getTicker());
  136. 136. ps.setString(2,portfolio.getName());
  137. 137. }
  138. 138. public int getBatchSize() {
  139. 139. return listOfTickersToDelete.size();
  140. 140. }
  141. 141. }); </li></ul>
  142. 142. <ul>CQL and JDBC </ul><ul><li>Summary:
  143. 143. - More than a proof of concept
  144. 144. - Not quite production ready
  145. 145. - Rough edges remain
  146. 146. - Ready for experimentation, test drives and bug hunting
  147. 147. *** Lot's of folks are now/will always be happy with the existing Thrift API </li></ul>
  148. 148. <ul>Data Model </ul><ul>GOOG </ul><ul>AAPL </ul><ul>NFLX </ul><ul>NOK </ul><ul><li>price: 589.55 </li></ul><ul><li>price: 401.76 </li></ul><ul>price: 78.73 </ul><ul>name : Google </ul><ul>name : Apple </ul><ul>name : Netflix </ul><ul>price: 6.90 </ul><ul>name : Nokia </ul><ul>exchange : NYSE </ul><ul>Portfolio </ul>
  149. 149. <ul>Data Model </ul><ul>GOOG </ul><ul>AAPL </ul><ul>NFLX </ul><ul>NOK </ul><ul><li>price: 589.55 </li></ul><ul><li>price: 401.76 </li></ul><ul>price: 78.73 </ul><ul>name : Google </ul><ul>name : Apple </ul><ul>name : Netflix </ul><ul>price: 6.90 </ul><ul>name : Nokia </ul><ul>exchange : NYSE </ul><ul>Stocks </ul>
  150. 150. <ul>Data Model </ul><ul>StockHist </ul><ul>10/25/2011: 6.71 </ul><ul>GOOG </ul><ul>AAPL </ul><ul>NFLX </ul><ul>NOK </ul><ul>10/24/2011: 6.76 </ul><ul>10/21/2011: 6.61 </ul><ul>10/25/2011: 77.37 </ul><ul>10/24/2011: 118.84 </ul><ul>10/21/2011: 117.04 </ul><ul>10/25/2011: 397.77 </ul><ul>10/24/2011: 405.77 </ul><ul>10/21/2011: 392.87 </ul><ul>10/25/2011: 583.16 </ul><ul>10/24/2011: 596.42 </ul><ul>10/21/2011: 590.49 </ul>
  151. 151. <ul>CQL via JDBC: Components </ul><ul><li>- HCQLDataSource (from jdbc-pool)
  152. 152. - Spring Framework's JdbcTemplate
  153. 153. - DAO class with associated domain objects
  154. 154. - Junit
  155. 155. - Spring Framework's SpringJUnit4ClassRunner (context setup and injection)
  156. 156. - EmbededServerHelper from hector-test (manage Cassandra lifecycle, directories and configuration) </li></ul>
  157. 157. <ul>Development Resources </ul>CQL Documentation (and CQL Shell) http://www.datastax.com/docs/1.0/dml/using_cql Hector Documentation http://hector-client.org <ul><li>Cassandra Maven Plugin (exec-cql goal) http://mojo.codehaus.org/cassandra-maven-plugin/
  158. 158. CCM localhost cassandra cluster https://github.com/pcmanus/ccm
  159. 159. OpsCenter http://www.datastax.com/products/opscenter </li></ul><ul>Cassandra AMIs https://github.com/riptano/CassandraClusterAMI </ul>
  160. 160. <ul>Questions </ul><ul>Nate McCall [email_address] @zznate </ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×