Add (Syntactic) Sugar To Your Java

5,815 views

Published on

Abstract: kaChing powers the largest social investment site on the web with nearly 500,000 registered users. Our mission is to make the investment world open by offering transparent investment vehicles that directly compete with mutual funds.

Over the past year and a half, we have built a large feature set and evolved our software continuously with very short iterations and (almost) no regression. In this talk, I will present our experience building a large test-driven code base from the ground up. Using concrete examples, we will have a look at component based APIs, declarative programming, minimizing the concepts of an API, specific cases of separation of concern and interactions with third-party software. We will look at multiple programming paradigms from languages such as Scala, shell script and Prolog and see how these ideas can be embedded as syntactic sugar in your Java.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,815
On SlideShare
0
From Embeds
0
Number of Embeds
589
Actions
Shares
0
Downloads
52
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • - syntactic sugar refers to convenience features implemented at the urface level of a programming language, for instance i++ in Java is a short hand for increment by 1 plus a cast or foreach added in Java 5 to avoid dealing with iterators directly -in our experience, many of the code we write is to some extent an extension of the language , basically creating a Domain Specific Language to solve a particular problem -talk about how we implemented things rather than what we implemented -cover multiple examples , including our platform the QueryEngine -effort of a whole team presented, many contributions and arguments about how things should be built
  • Add (Syntactic) Sugar To Your Java

    1. 1. Add Sugar to your Java “ how we architected from the ground up for testability, and why” Pascal-Louis Perez – .com Pascal-Louis Perez, kaChing Group Inc. V SYNTACTIC
    2. 2. .com Pascal-Louis Perez, kaChing Group Inc.
    3. 3. our system in 30 seconds Pascal-Louis Perez, kaChing Group Inc.
    4. 4. Overview <ul><li>Components versus Layers </li></ul><ul><li>Type Driven Development </li></ul><ul><ul><li>1:1 principle </li></ul></ul><ul><ul><li>Family Polymorphism </li></ul></ul><ul><li>Annotations </li></ul><ul><ul><li>Meta Information </li></ul></ul><ul><ul><li>Typing Extensions </li></ul></ul><ul><li>Object Oriented design replaces Conditionals </li></ul><ul><ul><li>Delegation </li></ul></ul><ul><ul><li>State Pattern </li></ul></ul><ul><li>Declarative Programming </li></ul><ul><ul><li>Domain Specific Language </li></ul></ul><ul><ul><li>Factories </li></ul></ul>Pascal-Louis Perez, kaChing Group Inc. τ @ what?
    5. 5. µ-Java <ul><li>Imagine Java deprived of all native types ( boolean , int , …) and all control-flow constructs ( if then else , for , while , …) </li></ul><ul><li>How do you encode booleans using only objects? </li></ul><ul><li>What about integers? </li></ul><ul><li>Is µ-Java Turing-complete? </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    6. 6. µ-Java: boolean <ul><li>if (condition) statement1 else statement2 </li></ul><ul><li>condition.if(statement1, statement2) </li></ul><ul><li>Bool class with True and False subclasses </li></ul><ul><ul><li>in True, if computes statement1 </li></ul></ul><ul><ul><li>In False, if computes statement2 </li></ul></ul><ul><li>Statements are instances of Thunk<Void> </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    7. 7. Growing a Language Pascal-Louis Perez, kaChing Group Inc. expressiveness µ-Java 1 Do not take things for granted 2 Think outside the Language
    8. 8. Testable Code Because… <ul><li>It allows quick iterations </li></ul><ul><li>It makes it easy to scale the team </li></ul><ul><li>It is more cost effective than debugging </li></ul><ul><li>It obsoletes the need for functional QA </li></ul><ul><li>It facilitates continuous refactoring, allowing the code to get better with age </li></ul><ul><li>It attracts the right kind of engineers </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    9. 9. It’s a lot of fun! Pascal-Louis Perez, kaChing Group Inc.
    10. 10. And it can be very geeky! Pascal-Louis Perez, kaChing Group Inc.
    11. 11. Software is Traditionally Layered <ul><li>Examples </li></ul><ul><ul><li>Jetty </li></ul></ul><ul><ul><li>Apache HTTP client </li></ul></ul>Pascal-Louis Perez, kaChing Group Inc. STOP
    12. 12. Think about Components <ul><li>Provides Functionality… </li></ul><ul><li>given Required Services. </li></ul><ul><li>Yields reusable software </li></ul><ul><li>Key to testability </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    13. 13. Wireable Puzzle Pieces <ul><li>What if I want only one thread pool in my whole app server? </li></ul><ul><li>In Scala ( http://www.scala-lang.org ) new JettyServer with MyThreadPool; new ParallelRpc with MyThreadPool; </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    14. 14. Dependency Injection to the Rescue <ul><li>class UserServiceImpl </li></ul><ul><li>implements UserService { </li></ul><ul><li>@Inject </li></ul><ul><li>UserServiceImpl( HttpClient, … ) { … </li></ul><ul><li>} </li></ul>Pascal-Louis Perez, kaChing Group Inc. required services functionality τ
    15. 15. Add Retrying Logic <ul><li>class MyExtendedUserServiceImpl </li></ul><ul><li>implements UserService { </li></ul><ul><li>@Inject UserServiceImpl( UserServiceImpl ) … </li></ul><ul><li>User getUser (Id<User> id) { </li></ul><ul><li>User u = null; </li></ul><ul><li>for (int i=0; i<5 && u == null; i++) </li></ul><ul><li>u = delegate . getUser (id); </li></ul><ul><li>return u; </li></ul><ul><li>} </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    16. 16. QueryEngine <ul><li>Software platform to build services quickly </li></ul><ul><li>Unit of work is a Query </li></ul><ul><ul><li>query constructors define input parameters </li></ul></ul><ul><ul><li>queries can request dependencies at run-time </li></ul></ul><ul><ul><li>queries do work and produce a value which is then post-processed (serialized, stored, …) </li></ul></ul><ul><li>Soon open-sourced http://code.google.com/p/queryengine/ </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    17. 17. Shell’s “everything is a process” Pascal-Louis Perez, kaChing Group Inc. process stdin args stdout stderr return code
    18. 18. Shell is a plumbing language <ul><li>if echo $name | grep –q “Jack”; then </li></ul>Pascal-Louis Perez, kaChing Group Inc. echo $name $name grep -q return code if return code
    19. 19. Queries Pascal-Louis Perez, kaChing Group Inc. query dependencies exception arguments value
    20. 20. Query Constructors <ul><li>Similar to dependency injection, you ask for the arguments you need rather than fetch them from the environment </li></ul><ul><li>Type safe regardless of input format </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    21. 21. Query Constructors <ul><li>AddToWatchlist(Id<User>, Symbol) </li></ul><ul><li>AddToGroup(Id<User>, Id<Group>, @Optional boolean) </li></ul><ul><li>PlaceOrder(Id<User>, Action, Symbol, @Positive int) </li></ul><ul><li>PostMessage(Id<User>, Id<Wall>, Text<Plain>) </li></ul>Pascal-Louis Perez, kaChing Group Inc. @ τ
    22. 22. HelloWorld Query <ul><li>class HelloWorld extends AbstractQuery<String> { </li></ul><ul><li>HelloWorld(String name) { … </li></ul><ul><li>@Inject DateTime now; </li></ul><ul><li>String process() { </li></ul><ul><li>return now + “: Hello ” + name; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    23. 23. QueryDriver Pascal-Louis Perez, kaChing Group Inc. @ query monitoring retrying transacting injecting scoping V LazyManager
    24. 24. Adapt Queries <ul><li>Backend servers </li></ul><ul><li>Frontend server for GWT </li></ul><ul><li>Command-line tools </li></ul><ul><li>Tests </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    25. 25. Conditionals Complicate Testing <ul><li>if (a) { </li></ul><ul><li>if (b) { </li></ul><ul><li>section 1 </li></ul><ul><li>} else { </li></ul><ul><li>section 2 </li></ul><ul><li>} </li></ul><ul><li>} else { </li></ul><ul><li>if (b) { </li></ul><ul><li>section 3 </li></ul><ul><li>} else { </li></ul><ul><li>section 4 </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>Branch coverage requires a lot of tests </li></ul><ul><li>Typically, deep conditionals require a lot of obscure setup in tests </li></ul><ul><li>Ties more closely the test and the implementation </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    26. 26. Thunk<T> <ul><li>abstract class Thunk<T> { </li></ul><ul><li>private boolean evaluated = false; </li></ul><ul><li>private T value; </li></ul><ul><li>public T get() { </li></ul><ul><li>if (!evaluated) { </li></ul><ul><li>value = compute(); </li></ul><ul><li>evaluated = true; </li></ul><ul><li>} </li></ul><ul><li>return value; </li></ul><ul><li>} </li></ul><ul><li>abstract protected T compute(); </li></ul><ul><li>} </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    27. 27. Thunk<T> without conditional <ul><li>abstract class Thunk<T> { </li></ul><ul><li>private Getter value = new Getter<T>() { </li></ul><ul><li>public T get() { </li></ul><ul><li>final T v = compute(); </li></ul><ul><li>value = new Getter<T>() { </li></ul><ul><li>T get() { return v; } </li></ul><ul><li>}; </li></ul><ul><li>return value .get(); </li></ul><ul><li>} </li></ul><ul><li>}; </li></ul><ul><li>public T get() { return value .get(); } </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    28. 28. JsonMarshaller <ul><li>Marshalling library converting Java POJOs into JSON and vice-versa. </li></ul><ul><li>Annotations and persistence ideas inspired by Hibernate </li></ul><ul><li>“ kaChing’s proto buffers” </li></ul><ul><li>http://code.google.com/p/jsonmarshaller/ </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    29. 29. JsonMarshaller <ul><li>@Entity </li></ul><ul><li>class Googler { @Value List<Cafe> favoriteCafes; </li></ul><ul><li>} </li></ul><ul><li>createMarshaller(Googler.class).marshall(…); </li></ul><ul><li>produces </li></ul><ul><li>{“favoriteCafes”:[“150”, “Charlie’s”]} </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    30. 30. JsonMarshaller <ul><li>@Value ABC enumAbc; </li></ul><ul><li>@Value(ordinal = true) ABC enumAbc; </li></ul><ul><li>dedicated code is wired at creation time </li></ul><ul><li>(almost) no conditionals at run-time </li></ul>Pascal-Louis Perez, kaChing Group Inc. @
    31. 31. JsonMarshaller <ul><li>Conditionals </li></ul><ul><li>if(annotation.ordinal) </li></ul><ul><li>marshallOrdinal() </li></ul><ul><li>else </li></ul><ul><li>marshallName() </li></ul><ul><li>Delegation </li></ul><ul><li>marshall() </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    32. 32. kaChing’s API REST path parsing <ul><li>REST API with paths such as /users/ :id: /users/ :id: /portfolio/trades/ :tradeId: /communities/everyone/insight </li></ul><ul><li>HTTP parameters </li></ul><ul><li>Headers (X-KaChing-ApiKey, X-KaChing-RemoteKey) </li></ul><ul><li>GET, PUT, POST, DELETE </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    33. 33. Express “ what ” instead of Coding “ how ” <ul><li>path( </li></ul><ul><li>&quot;users&quot;, </li></ul><ul><li>path( </li></ul><ul><li>FetchUser.class, </li></ul><ul><li>path( </li></ul><ul><li>&quot;watchlist&quot;, </li></ul><ul><li>QuerySpec. of( </li></ul><ul><li>FetchUser.class, </li></ul><ul><li>a sList(UserPart.WATCHLIST)), </li></ul><ul><li>path( immutableMap( </li></ul><ul><li>PUT, AddToWatchlist.class, </li></ul><ul><li>DELETE, RemoveFromWatchlist.class)))), </li></ul>Pascal-Louis Perez, kaChing Group Inc. τ /users/ :id: handled by FetchUser.class /users/ :id: /watchlist handled by FetchUser.class with the extra paramater WATCHLIST PUT /users/ :id: /watchlist handled by AddToWatchlist.class what?
    34. 34. I’ll have a big boilerplate for lunch <ul><li>Portfolio p = new Portfolio(…); </li></ul><ul><li>Stock ko = new Stock(…); </li></ul><ul><li>p.executeTrade( </li></ul><ul><li>new BuyStockTrade(ko, …)); </li></ul><ul><li>p.executeTrade(new Interest(…)); </li></ul><ul><li>p.executeTrade( </li></ul><ul><li>new SellStockTrade(ko, …)); </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    35. 35. … or maybe not Pascal-Louis Perez, kaChing Group Inc. portfolio(now, 90) .buyStock(now, &quot;KO&quot;, 10, 1) .payInterest(later, fedRateRepository) .sellStock(latest, &quot;KO&quot;, 10, 1) .get() what?
    36. 36. Find Managers Pascal-Louis Perez, kaChing Group Inc. order constraint
    37. 37. Constraint, Index, Selector <ul><li>Constraint: search criteria on one dimension </li></ul><ul><ul><li>e.g. age = 4 </li></ul></ul><ul><li>Index: search structure </li></ul><ul><ul><li>can handle only one kind of constraint </li></ul></ul><ul><li>Selector: constraint template used to find appropriate index </li></ul><ul><ul><li>e.g. age = ? </li></ul></ul>Pascal-Louis Perez, kaChing Group Inc. dimension 1 dimension 2 … order 1 order 2 …
    38. 38. Practice Safe Programming: Use Types Pascal-Louis Perez, kaChing Group Inc. Selector createIndex : Index Index search( Constraint) Constraint getSelector : Selector
    39. 39. Practice Safe Programming: Use Types Pascal-Louis Perez, kaChing Group Inc. τ Selector <M> createIndex : Index <M> Index <M> search( Constraint <M> ) Constraint <M> getSelector : Selector <M>
    40. 40. Summary of the Examples Pascal-Louis Perez, kaChing Group Inc. τ @ dependency injection query constructors query driver thunk JsonMarshaller API paths parsing builder pattern family polymorphism ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ µ-Java ✓ what?
    41. 41. Key Takeaways <ul><li>Build components instead of providing extension points </li></ul><ul><li>Apply the 1:1 principle </li></ul><ul><li>Simplify your life with annotations </li></ul><ul><li>Use delegation instead of conditionals </li></ul><ul><li>Languages are conveniences, not barriers </li></ul>Pascal-Louis Perez, kaChing Group Inc.
    42. 42. Further Readings <ul><li>TotT: Too Many Tests </li></ul><ul><li>http://googletesting.blogspot.com/2008/02/in-movie-amadeus-austrian-emperor.html </li></ul><ul><li>TotT: The Invisible Branch </li></ul><ul><li>http://googletesting.blogspot.com/2008/05/tott-invisible-branch.html </li></ul><ul><li>JsonMarshaller http://code.google.com/p/jsonmarshaller/ </li></ul><ul><li>Real-World Scala: Dependency Injection http://jonasboner.com/2008/10/06/real-world-scala-dependency-injection-di/ </li></ul><ul><li>Scalable Component Abstraction http://lamp.epfl.ch/~odersky/papers/ScalableComponent.pdf </li></ul><ul><li>Unifying Type Parameters in Java http://blog.kaching.com/index.php/2009/01/16/unifying-type-parameters-in-java/ </li></ul><ul><li>Japex: Micro-Benchmarking https://japex.dev.java.net/ </li></ul><ul><li>The Testability Explorer, Misko Hevery http://misko.hevery.com </li></ul><ul><li>Growing a Language, by Guy Steele http://video.google.com/videoplay?docid=-8860158196198824415 </li></ul><ul><li>Type-Safe Builder Pattern in Scala http://blog.rafaelferreira.net/2008/07/type-safe-builder-pattern-in-scala.html </li></ul>Pascal-Louis Perez, kaChing Group Inc.

    ×