The document discusses testing Cassandra applications using Stubbed Cassandra, a tool that allows priming Cassandra to return specific responses and errors to integration tests. It provides an overview of Stubbed Cassandra's features like starting an embedded server, priming queries and prepared statements, verifying query executions, and testing error scenarios. Examples are given of how to test different behaviors like retries, timeouts, and coordinator issues. The speaker encourages helping with the open source project and answers questions.
2. @chbatey
Who am I?
• Based in London
• Technical Evangelist for Apache Cassandra
•Work on Stubbed Cassandra
•Help out Apache Cassandra users
• Previous: Cassandra backed apps at BSkyB
11. @chbatey
Write timeout
• Received acknowledgements
• Required acknowledgements
• Consistency level
• CAS and Batches are more complicated:
- WriteType: SIMPLE, BATCH, UNLOGGED_BATCH,
BATCH_LOG, CAS
12. @chbatey
Batches
• BATCH_LOG
- Timed out waiting for batch log replicas
- Retry if you like
• BATCH
- Written to batch log but timed out waiting for actual replica
- Will eventually be committed (serial read)
• UNLOGGED_BATCH
- Unknown - retry if you like
17. @chbatey
Unit & Integration testing
• Requirements: Quick to run, deterministic, not brittle!!
• Integration tests against an embedded Cassandra?
• cassandra-unit
• Integration tests against an external Cassandra running on developer / CI
machines
• CCM
• Regular install
• Docker
• Vagrant
• Mocking the driver
19. @chbatey
How do this if it was a HTTP service?
• Release it - great book
• Wiremock - mocking HTTP
services
• Saboteur - adding network
latency
If you want to build a fault tolerant application
you better test faults!
21. @chbatey
Two sides of Scassandra
• Java
-Acts as a unit/integration testing library
-90% of effort on this side
• Standalone
-Runs as a separate process
22. @chbatey
Java Client
• Embeds Stubbed Cassandra in a Java library
- Start / stop the server
- Prime the server to return rows and/or errors
- Verify exactly the queries your application has executed
23. @chbatey
Starting / Stopping
@ClassRule
public static final ScassandraServerRule SCASSANDRA = new
ScassandraServerRule();
Scassandra scassandra = ScassandraFactory.createServer();
scassandra.start();
PrimingClient primingClient = scassandra.primingClient();
ActivityClient activityClient = scassandra.activityClient();
24. @chbatey
Activity Client
• Query
‣ Query text
‣ Consistency
• PrepreparedStatementExecution
‣ Prepared statement text
‣ Bound variables
public List<Query> retrieveQueries();
public List<PreparedStatementExecution> retrievePreparedStatementExecutions()
public List<Connection> retrieveConnections();
public void clearAllRecordedActivity()
public void clearConnections();
public void clearQueries();
public void clearPreparedStatementExecutions();
25. @chbatey
Priming Client
• PrimingRequest
‣ Either a Query or PreparedStatement
‣ Query text or QueryPattern (regex)
‣ Consistency (default all)
‣ Result (success, read timeout, unavailable etc)
‣ Rows for successful response
‣ Column types for rows
‣ Variable types for prepared statements
public void prime(PrimingRequest prime)
public List<PrimingRequest> retrievePreparedPrimes()
public List<PrimingRequest> retrieveQueryPrimes()
public void clearAllPrimes()
public void clearQueryPrimes()
public void clearPreparedPrimes()
26. @chbatey
Example time
public interface PersonDao {
void connect();
void disconnect();
List<Person> retrievePeople();
List<Person> retrievePeopleByName(String firstName);
void storePerson(Person person);
}
27. @chbatey
Testing the connect method
@Test
public void shouldConnectToCassandraWhenConnectCalled() {
//given
activityClient.clearConnections();
//when
underTest.connect();
//then
assertTrue("Expected at least one connection",
activityClient.retrieveConnections().size() > 0);
}
28. @chbatey
Testing retrieve
public List<Person> retrieveNames() {
ResultSet result;
try {
Statement statement = new SimpleStatement("select * from person");
statement.setConsistencyLevel(ConsistencyLevel.QUORUM);
result = session.execute(statement);
} catch (ReadTimeoutException e) {
throw new UnableToRetrievePeopleException();
}
List<Person> people = new ArrayList<>();
for (Row row : result) {
people.add(new Person(row.getString("first_name"), row.getInt("age")));
}
return people;
}
29. @chbatey
Test the query + consistency
@Test
public void testQueryIssuedWithCorrectConsistencyUsingMatcher() {
//given
Query expectedQuery = Query.builder()
.withQuery("select * from person")
.withConsistency("QUORUM").build();
//when
underTest.retrievePeople();
//then
assertThat(activityClient.retrieveQueries(), containsQuery(expectedQuery));
}
Hamcrest matcher
30. @chbatey
Testing behaviour
@Test
public void testRetrievingOfNames() throws Exception {
// given
Map<String, ?> row = ImmutableMap.of(
"first_name", "Chris",
"last_name", "Batey",
"age", 29);
primingClient.prime(PrimingRequest.queryBuilder()
.withQuery("select * from person")
.withColumnTypes(column("age", PrimitiveType.INT))
.withRows(row)
.build());
//when
List<Person> names = underTest.retrievePeople();
//then
assertEquals(1, names.size());
assertEquals(“Chris Batey", names.get(0).getName());
}
Each Cassandra Row == Java map
Tell Scassandra about your schem
35. @chbatey
Testing slow connection
@Test(expected = UnableToSavePersonException.class)
public void testThatSlowQueriesTimeout() throws Exception {
// given
PrimingRequest preparedStatementPrime = PrimingRequest.preparedStatementBuilder()
.withQueryPattern("insert into person.*")
.withVariableTypes(VARCHAR, INT, list(TIMESTAMP))
.withFixedDelay(1000)
.build();
primingClient.prime(preparedStatementPrime);
underTest.connect();
underTest.storePerson(new Person("Christopher", 29, Collections.emptyList()));
}
Delays the response by 1000ms
Expect a custom exception
36. @chbatey
Testing slow connection
SocketOptions socketOptions = new SocketOptions();
socketOptions.setReadTimeoutMillis(500);
cluster = Cluster.builder()
.addContactPoint("localhost")
.withPort(port)
.withRetryPolicy(new RetryReads())
.withSocketOptions(socketOptions)
.build();
Set a read timeout
@Override
public void storePerson(Person person) {
try {
BoundStatement bind = storeStatement.bind(person.getName(), person.getAge());
session.execute(bind);
} catch (NoHostAvailableException e) {
throw new UnableToSavePersonException();
}
}
37. @chbatey
Summary of features
• Start as many instances as you like in the same JVM
• Prime all primitive types + collections
• Prime prepared statements
• Verify number of connections your application makes
• Test slow queries
39. @chbatey
How do I get it?
•It is on maven central
•Or go to www.scassandra.org
•Executable jar + REST API
<dependency>
<groupId>org.scassandra</groupId>
<artifactId>java-client</artifactId>
<version>0.6.0</version>
<scope>test</scope>
</dependency>
40. @chbatey
I want to help
• Server: https://github.com/scassandra/scassandra-
server
• Client: https://github.com/scassandra/scassandra-java-
client
• Tests: https://github.com/scassandra/scassandra-it-java-
driver-2
41. @chbatey
Summary
• Testing is good - I want more tools to help me
• Existing tools for Cassandra are great at happy path
scenarios
• Stubbed Cassandra allows testing of edge cases
42. @chbatey
The end - Questions?
www.scassandra.org
http://christopher-batey.blogspot.co.uk/
Ping me on twitter @chbatey
43. @chbatey
How does this work?
• Server implemented in Scala
• Binary port for your Cassandra application to connect to
• Admin port for the REST API
• Thin Java wrapper to make it easy to be used in JUnit
tests