Extreme Testing at kaChing

Extreme Testing at kaChing
From Commit to Production in 5 minutes

Pascal‐Louis Perez – .com

Pascal‐Louis Perez, kaChing Group Inc.

kaChing: the Investing Talent Marketplace

• Marketplace for qualified managers
– Young professionals
– RIAs
– Hedge Fund managers
• Customers can mirror “kaChing Geniuses”
– Your own brokerage account
– Real time reporting; Analytics; Drift alerts
• Unparalleled Transparency
– See everything: holdings, risk metrics, trades, …
– Clear fee structure: management fee. No hidden fees,
exit fees, …

Our System in 30 Seconds


Testing Matters

Architectural Soundness
Delighting Customers


Testable Code Because…
• It allows quick iterations
• It makes it easier to scale the team
• It is more cost effective than debugging
• It obsoletes the need for functional QA
• It facilitates continuous refactoring, allowing
the code to get better with age
• It attracts the right kind of engineers


High Level Classification

Correctness Availability

λ

π




λ • Bad: incorrect but available system
• Bad: correct but unavailable system

π



• Transformations (λ)
λ • Interactions (π): send an e‐mail,
store data in a db, RPC call
• In π implicit global mutable state

π




λ
• Simplest to test
• Input produces an output: just assert
π • (Not to be confused with
implementation complexity!)




λ
• Transformations can only be CPU
π bound
• Timing tests, benchmarking, …




λ

π
• Key idea: fake the world
• Mocks, stubs, fakes




λ

π
• Work in small batches
• End‐to‐end tests, regression test,
performance lab
(Find yourself a great architect!)


Think Outside the Box
C A
λ
π

A C A
C
λ λ

π π
C A
λ
π


Repository (or Client) Pattern

C A @ImplementedBy(DbUserRepo.class)
λ interface UserRepository {
π User get(Id<User> id);
}

UserRepository repo = ...;
User pascal = repo.get(Id.<User> of(8));

C A class DbUserRepo
λ implements UserRepository {
π // implementation
}


Components
• Software composability is key to test‐driven
development
• Testable software abstracts the control flow;
As such is functional by nature


Examples of Simple Tests
• Marshalling tests
• Dealing with time
• Equivalence tester


C A
λ

Marshalling tests
π

Json.Object o =  Fail fast
TwoLattes.createMarshaller(Trade.class).marshall(...);

assertSameJson(
object(
string("id"), number(4),
string("portfolioId"), NULL,
string("trade"), string("interest"),
string("date"), string("2008-07-06"),
string("toDate"), string("2008-06-06"),
string("fromDate"), string("2008-07-05"),
string("longInterest"), number(56.43),
string("shortInterest"), number(-0.81)),
o);


C A
λ

Marshalling tests
π

Json.Object o =

assertSameJson(  Dedicated assertions
object(
o);


C A
λ

Marshalling tests
π

Json.Object o =

assertSameJson(  Descriptive expectation
object(
o);


C A
λ

Dealing with time
π

• Use clocks
– Provider<DateTime>
– Provider<LocalDate>
• Pass in now or today in business logic


Time Logic

@VisibleForTesting
<T> T provideCurrentOrHistorical(
DateTime date, Provider<T> current, Provider<T> historical) {
DateTime lastTradeDate = getLastTradeDate();  Time logic is easy
boolean isCurrent = date == null to test
|| lastTradeDate == null
|| date.compareTo(lastTradeDate) > 0;
return isCurrent ? current.get() : historical.get();
}


Time Logic

@VisibleForTesting
<T> T provideCurrentOrHistorical(  Higher‐order functions
DateTime date, Provider<T> current, Provider<T> historical) {
DateTime lastTradeDate = getLastTradeDate();
boolean isCurrent = date == null
|| lastTradeDate == null
|| date.compareTo(lastTradeDate) > 0;
return isCurrent ? current.get() : historical.get();
}


C A
λ

Equivalence Tester
π

EquivalenceTester.check( Tests
asList(  Reflexivity
new Isin("abCdEFgHiJK3“),  Symmetry
new Isin("ABCDEFGHIJK3“),  Transitivity
new Isin(“abCDefGHIJK3")),  hashCode consistency
asList(
new Isin("AU0000XVGZA3"),
new Isin("AU0000XVGZA3")));


Rationale for Database Testing
• Accessing the database correctly
• O/R properly configured and used
• Hand written queries
• Optimistic concurrency


JDBC
 Global static state
DriverManager.registerDriver(new org.gjt.mm.mysql.Driver());
Connection conn = DriverManager.getConnection("jdbc:...");
PreparedStatement st = conn.prepareStatement("select ...");
st.setString(1, "...");
ResultSet rs = st.executeQuery();
while (rs.next()) {
// work
}
rs.close();
st.close();
conn.close();


JDBC

while (rs.next()) {
// work
}
rs.close();
 Low level, stateful API, wide scoping
st.close();
conn.close();


JDBC

while (rs.next()) {
// work
}  Resources hell
rs.close();
st.close();
conn.close();


Hibernate
 Global state
SessionFactory factory = ...;
Session session = factory.openSession();
Transaction tx = session.beginTransaction();
List<User> users = session
.createQuery("from User where ...")
.list();
for (User user : users) {
// work
}
tx.commit();
session.close();


Hibernate

SessionFactory factory = ...;  High‐level API
.list();
// work
}
tx.commit();
session.close();


Hibernate

SessionFactory factory = ...;
.list();
// work
}  Resources hell
tx.commit();
session.close();


Transacter
 Global state
Transacter transacter = ...;
transacter.execute(new WithSession() {
public void run(Session session) {
List<User> users = ...;
// work
}
}
});


Transacter

Transacter transacter = ...;  High‐level API, no resources
public void run(Session session) {
// work
}
}
});


Transacter

Transacter transacter = ...;
public void run(Session session) {  Lexical scoping
// work
}
}
});


Database Testing is Hard
• Complex to set up : PersistenceTestRunner 
• Schema installation : O/R Mapping 
• Requires data to be loaded : Fixtures, DbUnit 
• Developer must have proper environment
: in‐memory DBs (HSQLDB, Connector/MXJ) 


C A
λ

PersistenceTestRunner
π

@RunWith(PersistenceTestRunner.class)  Declarative setup
@PersistentEntities({Item.class, Note.class})
public class ExamplePersistentTest {

@Test
public void example1(Transacter transacter) {
// work
}

@Test
public void example2() {
// work
}

}


C A
λ

π

@RunWith(PersistenceTestRunner.class)
 Lexical scoping
@Test
// work
}

@Test
// work
}

}


C A
λ

π


@Test  RuntimeException
// work
}

@Test
// work
}

}


C A
λ

π


@Test
// work
}
 Non‐persistent tests run as usual
@Test
// work
}

}


C A
λ

Testing Interactions (I/O, …)
π

• Implicit global state
• Many cases
– HTTP GET: succeed, fail to connect, connection
closed before headers are read, …
– Sending e‐mail: succeed, connection lost after
HELO, …
• Sequencing is crucial


C A
λ

Testing the Transacter (success)
π

 Mocking
final WithSession withSession = mockery.mock(WithSession.class);
mockery.checking(new Expectations() {{
one(sessionFactory).openSession();
will(returnValue(session));
inSequence(execution);
one(session).connection(); will(...); inSequence(...);
one(connection).setReadOnly(with(equal(false))); ...
one(connection).setAutoCommit(with(equal(false))); ...
one(session).beginTransaction(); inSequence(...);
...
}});

transacter.execute(withSession);

mockery.assertIsSatisfied();


C A
λ

Testing the Transacter (success)
π

final WithSession withSession = mockery.mock(WithSession.class);
 Interactions are scripted
will(returnValue(session));
one(session).connection(); will(...); inSequence(...);
one(connection).setReadOnly(with(equal(false))); ...
one(connection).setAutoCommit(with(equal(false))); ...
one(session).beginTransaction(); inSequence(...);
...
}});

transacter.execute(withSession);



C A
λ

Testing the Transacter (failure)
π

 Failure is controlled
...

one(connection).setReadOnly(with(equal(false)));
will(throwException(exception));

one(session).close(); inSequence(execution);
}});

try {
transacter.execute(withSession); fail();
} catch (RuntimeException e) {
assertTrue(e == exception);
}

C A
λ

π

...


}});

try {
assertTrue(e == exception);  Exact same object (correct bubbling)
}

C A
λ

π

...


}});
 Unreachable program point
try {
assertTrue(e == exception);
}

Meta Tests
• Cross domain tests to sweep the codebase,
exactly like static analysis
• Catch common mistakes
– Distracted developers
– Bad APIs
– New hires learning conventions


C A
λ

Dependency Test
π

• Declare software dependencies
• Controls layering of application
• Simpler and more flexible than different
compilation units
• Can guarantee that certain APIs are not used
directly
• Patched Jdepend to parse annotations


Dependency Test
dependencies.forPackages(
"com.kaching.*"
).

check("com.kaching.supermap").mayDependOn(
"com.kaching.platform.guice",
"org.apache.commons.lang",
"voldemort.*",
"org.kohsuke.args4j"
).

assertIsVerified(jDepend.getPackages());


C A
λ

Bad Code Test
π

• Enforce conventions
• Gotchas, bad APIs
– E.g. System.out.println
– E.g. java.net.URL
• Code reviews
– Design patterns, custom APIs, etc.
– Enforce conventions: automate this!


Bad Code Test
@RunWith(BadCodeSnippetsRunner.class)
@CodeSnippets({
@Check(paths = "src/com/kaching", snippets = {

// no uses of java.net.URL
@Snippet(value = "bjava.net.URLb", exceptions = {
"src/com/kaching/...",
"src/com/kaching/..."
}),

// never call default super constructor
@Snippet("super()")
})})
class MyBadCodeSnippetsTest {


Key Takeaways
• Invest in your tests, and testing frameworks
• Design patterns to separate concerns
• Components
• TDD pushes towards functional techniques

• We’re recruiting jobs@kaching.com
• More http://eng.kaching.com


Extreme Testing at kaChing

Recommended

Recommended

More Related Content

Similar to Extreme Testing at kaChing

Similar to Extreme Testing at kaChing (20)

More from Pascal-Louis Perez

More from Pascal-Louis Perez (12)

Recently uploaded

Recently uploaded (20)

Extreme Testing at kaChing