Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JavaOne 2017 - Choosing a NoSQL API and Database to Avoid Tombstones and Dragons in Game On! [CON1756]


Published on

Game On! (@gameontext – is an awesome throwback text-based adventure built with microservices. Completely open source, it enables everyone to choose their own adventure to learn about microservices concepts while extending the game. One of the core services is the Map, which maintains a two-dimensional map containing all the registered rooms. The Map started with a document store as a back end, but as the Map changed over time, tombstones started to accrue. And then people started to ask how to manage three dimensions, and dragons appeared. Come to this session to find out why the decision was made to change the NoSQL back end, how it was done, and the result of the change with a new NoSQL API (

Published in: Technology
  • Login to see the comments

JavaOne 2017 - Choosing a NoSQL API and Database to Avoid Tombstones and Dragons in Game On! [CON1756]

  1. 1. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Choosing a NoSQL API and Database to Avoid Tombstones and Dragons in Game On! Leonardo Lima Otávio SantanaKate Stanley
  2. 2. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava About the Speakers Katherine (Kate) Stanley • Software Engineer, works for IBM • From United Kingdom • Work on Java development for IBM Cloud • Committer to Game On • Contributed to MicroProfile conference app • Co-wrote IBM Redbook about Microservices with Java
  3. 3. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava About the Speakers Leonardo Lima • Computer engineer, server & embedded sw developer • From São Paulo, Brasil, currently in Austin, TX • CTO at V2COM • Spec Lead – JSR363 – Units of Measurement • V2COM’s Representative at JCP Executive Committee []
  4. 4. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Otávio Santana • Software engineer, Tomitribe • From Salvador, Brazil • Java Champion, SouJava JUG Leader • Apache, Eclipse and OpenJDK Committer • Expert Group member in many JSRs • Representative at JCP EC for SouJava About the Speakers
  5. 5. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Microservices & GameOn!
  6. 6. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Microservices are used to… — … compose a complex application using: ◦ “small” ◦ independent (autonomous) ◦ replaceable ◦ processes — … that communicate via language-agnostic APIs Microservice MicroserviceMicroservice Microservice Microservices application Subscribe REST (JSON/HTTP) Microservice Microservice broker API Microservice Publish API REST (JSON/HTTP) Publish Service Registry API Microservice
  7. 7. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Microservices sample apps… Create a single service Rebuild a pre-baked microservices application Clueless No idea Confident Has read all the things! Experienced Hands-on understanding Puzzled / Realistic Challenges are real
  8. 8. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Enter… GameOn! — Hands on with microservices — Stick with ’HelloWorld’ simplicity — Choose your own adventure — Fast path to the hard stuff — Build something cool (to you!) — Learn as you go
  9. 9. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Twelve Factors — “a methodology for building software-as-a-service applications” ◦ Created by developers at Heroku — Factors are independent of ◦ programming language, ◦ backing services, ◦ cloud provider —
  10. 10. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava At first… ELK mongo room Player Proxy JSON/HTTP (REST) Concierge
  11. 11. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava … and now! kafkaELK amalgam8couchdb room PlayerAuth SweepProxystatic MediatorMap
  12. 12. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava About Dragons and Tombstones — Problem: Maintaining a centered, densely packed grid of rooms — SQL is hard. — CouchDB uses javascript to create custom indexes — BUT: Deleting rooms leaves tombstones.They never go away.
  13. 13. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava The Map API — Site has Name, Position (X,Y), Connection information to the WebService and a list of its Doors. — CRUD service; when you retrieve a room, you get a list of its adjacent rooms
  14. 14. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava The Map API { "name":"First Room", "connectionDetails":{ …, "target":"ws://secondroom:9008/barn/ws", … }, "fullName":"The First Room", "description":"A helpful room with doors in every possible direction.", "repositoryUrl":"", "doors":{ "n":"A knobbly wooden door with a rough carving or a friendly face", "w":"A fake wooden door with stickers of friendly faces plastered all over it", "s":"A warped wooden door with a friendly face branded on the corner", "e":"A polished wooden door with an inlaid friendly face", } }
  15. 15. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava The Map Internals, version 1 — Literally a copy of what’s exposed — Hard to store references to other rooms — Room pre-allocation and movement make it cumbersom — No way to (really) delete rooms (!)
  16. 16. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava — A Map is a Collection of Rooms connected thru Doors — X connected toY is… a graph!
  17. 17. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava A graph, really? First Room Play Room Basement Dungeon
  18. 18. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Take 1: Neo4J — Created rooms using Neo4J APIs and Cypher query language — No easy way to do CRUD operations ◦ Convert from object to Cypher using inspection — In the end, just changing who was holding our chain…
  19. 19. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Take 2: JNoSQL!
  20. 20. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava What is JNoSQL? — Mapping API - Artemis — Communication API - Diana — No lock-in — Divide and conquer DAO Communication Mapping JNoSQL
  21. 21. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Eclipse JNoSQL — Eclipse Foundation — Apache v2 + EPL 1.0 — API to each NoSQL type — Configurable — Extensible Communication Mapping
  22. 22. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava JNoSQL – Diana APIs Key/Value Document Column Family Graph
  23. 23. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Names & Definitions — Annotated Entities — Template — Repository — Configuration
  24. 24. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Annotaded Entities — MappedSuperclass — Entity — Column @Entity public class Site { @Id private Long id; @Column private String name; … }
  25. 25. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Template @Inject private GraphTemplate template; public Optional<Site> getRecentRoom() { return template .getTraversalVertex() .has("doorAvailable", true) .<Site>stream() .sorted(ORDER_WEIGHT) .findFirst(); }
  26. 26. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Repository public interface SiteRepository extends Repository<Site, Long> { Optional<Site> findByName(String name); } @Inject @Database(GRAPH) private SiteRepository repository;
  27. 27. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Repository @Transactional public void save(Site site) { if (isNull(site.getId())) { Optional<Site> siteByName = repository.findByName(site.getName()); siteByName.ifPresent(site::merge); }; } public Optional<Site> findByName(String name) { return repository.findByName(name); }
  28. 28. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava The future & possibilities
  29. 29. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Why a graph makes MUCH sense? — Current 2D map isVERY limiting ◦ With a graph we can go 3D! — Different kinds of objects can be tracked: ◦ Items can be made unique and migrate — Other doors are possible!
  30. 30. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Why a graph makes MUCH sense? First Room Play Room Basement Dungeon The One Ring Suite
  31. 31. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Get involved! — — — —
  32. 32. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava ? & !
  33. 33. GameOn!+JNoSQL #JavaOne #JNoSQL @KateStanley91 @leomrlima @otaviojava Thanks!