Dropwizard with MongoDB and Google Cloud

2,390 views
2,179 views

Published on

Latest source code for this project can be found here:
https://github.com/yunspace/dropwizard-mongodb-billapi

Original reveal.js slides here: http://slides.com/yunzhilin/dropwizard-mongodb

Published in: Software

Dropwizard with MongoDB and Google Cloud

  1. 1. Dropwizard MongoDBDropwizard MongoDB One day prototype usingOne day prototype using ,, andand Yun Zhi Lin DropwizardDropwizard MongoDBMongoDB Google CloudGoogle Cloud Yunspace.com +YunZhiLin @YunZhiLin
  2. 2. About - Yun Zhi LinAbout - Yun Zhi Lin Computer Science @UNSW, Applied Finance @Macquarie Joined a startup in 2002, then moved into Banking IT Moonlights as a Web 2.0 (nosql, json, mobile) enthusiast Loves Java without bloated frameworks or appservers This project source code in GitHub
  3. 3. SpecSpec End to end prototype of a Bill Aggregator that mimics vendor API. See below for detail specifications.
  4. 4. Data ModelData Model Providers - bill provider Premises - location being provisioned for Subscriptions - subscribe to bill by account/reference # Bill - matching subscriptions trigger notifications Notifications - Sent to subscription.callbackUrl with UrlForBillRetrieval FK: List(Subscriptions) FK: List(Subscriptions) FK: Provider.id, Premise.id, Bill.accountNumber, List(Bi FK: Provider.id, Subscription.referenceNumber FK: Provider, Premise, Subscription
  5. 5. ActorsActors Producer - API hosted on Google Compute Engine API Operations - Insert, List, Get, Remove, Path on all entities, except Notification and Bill. Consumer - Use Sample Data - JSON files provided Postman HTTP client
  6. 6. RequirementsRequirements 1. HTTP transport 2. JSON payload 3. Use compute engine 4. Authentication 5. API versioning 6. Ops-friendly 7. Data Validation 8. Batch considerations
  7. 7. DesignDesign
  8. 8. Frameworks ChoiceFrameworks Choice Node.js - full stack JSON, but lacking expertise Play! Framework - too complex for this exercise Dropwizard - simple, war-less and proven
  9. 9. Database ChoiceDatabase Choice Google Cloud Datastore - lacking expertise MongoDB - perfect for JSON
  10. 10. Dropwizard + MongoDB = Meet RequirementsDropwizard + MongoDB = Meet Requirements 1. HTTP - Jersey RESTful web services 2. JSON - Jackson with Mongo-Jack, fastest Java parser 3. Use compute engine - Both API jar and DB hosted on GCE 4. Authentication - OAuth2 or custom Providers 5. API versioning - annotation based url mappings 6. Ops-friendly - healchecks and Codahale metrics 7. Data Validation - Hibernate Validatior 8. Batch considerations - use JSON lists
  11. 11. PlanPlan Due to the time contraint, it's important to decide on the approach taken to plan out a successfull prototype: Start to End - build each component fully one by one, can only partially demonstrate the story Minimum Viable Product - selectively build only critical operations for all components. So that together they tell a full story
  12. 12. Scope AdjustmentScope Adjustment Authentication - excluded API versioning - excluded Batch - simplified to single JSON parsing Queuing - another consideration for the real world, excluded Notification - manually triggered.
  13. 13. Final Use CaseFinal Use Case Only the following Operations demonstrate the core processes of subscription, notification and bill retrieval: 1. create a subscriptoin 2. Manually trigger notification, where UrlForBillRetrieval = subscription.callbackURL + billID 3. retrieve a Bill PUT /subscription/ GET /subscription/trigger/{subscriptoinID}/{billID}/ GET /bill/?billID={billID}
  14. 14. ImplementationImplementation 1. Build Tool 2. Configuration 3. MongoDB 4. Healthcheck 5. JSON POJOs 6. Resources 7. Application Glue Build ToolBuild Tool
  15. 15. Gradle combines the best of Ant and Maven repositories { mavenLocal() mavenCentral() } project.ext { dropwizardVersion = '0.7.0-SNAPSHOT' mongojackVersion = '2.0.0-RC5' dropWizardConfig = './src/main/resources/billproto.y } dependencies { compile 'io.dropwizard:dropwizard-core:' + dropwizar compile 'org.mongojack:mongojack:' + mongojackVersio testCompile group: 'junit', name: 'junit', version: } run { args 'server', dropWizardConfig } ConfigurationConfiguration YAML
  16. 16. POJO mongo: host: ds039487.mongolab.com port: 39487 db: tycoon-mongo user: #### password: #### public class MongoConfiguration { @NotNull public String host; @Min(1) @Max(65535) @NotNull public int port; @NotNull public String db; @NotNull MongoDBMongoDB
  17. 17. public String user; @NotNull public String password; } Hosting - GCE option Connection JacksonDBCollection DB Import (must be list of single line json files) MongLab mongo = new Mongo(mongoConfig.host, mongoConfig.port); db = mongo.getDB(mongoConfig.db); db.authenticate(mongoConfig.user, mongoConfig.password.t bills = wrap(db.getCollection("bill"), Bill.class, Strin dbCursor = bills.find().is("_id", billID); if (dbCursor.hasNext()) { Bill result = cursor.next();} mongoimport --host --username --password --db --collecti HealthCheckHealthCheck
  18. 18. Code Healthy Not healthy: @Override protected Result check() throws Exception { mongo.getDB(dbname).getCollectionNames(); return Result.healthy(); } {"MongoHealthCheck":{"healthy":true},"deadlocks":{"healt {"MongoHealthCheck":{"healthy":false, "message":"not authorized for query on tycoon-mo "error":{"message":"not authorized for query on "stack":[...]JSON POJOsJSON POJOs
  19. 19. "deadlocks":{"healthy":true}}Use . Then apply simple annotations for Jackson and Hibernate Validatior. http://www.jsonschema2pojo.org/ @JsonInclude(JsonInclude.Include.NON_NULL) public class Notification { @Id public String notificationID; @NotNull public String subscriptionID; @NotNull public String notificationURLForBillDataRetrival; @NotNull public String notificationURLForBillImageRetrival; } ResourcesResources
  20. 20. NOTE: Use @Id for UUID, not @ObjectIdAnnotate opertion, metrics, url path and output format. Jackson automatically parse POJO into specified mediaType. @GET @Timed @Path("trigger/{subscriptionID}/{billID}") @Produces(MediaType.APPLICATION_JSON) public Notification triggerNotification( @PathParam("subscriptionID") String subscription @PathParam("billID") String billID) { ... notification = new Notification(); notification.notificationID = UUID.randomUUID().toSt notification.subscriptionID = subscriptionID; notification.notificationURLForBillDataRetrival = subscription.subscriptionCallBackURL + "?billID= return notification; }
  21. 21. Application GlueApplication Glue Putting all of the above together @Override public void run( BillprotoConfiguration configuration, Environment environment) throws Exception { MongoManaged mongoManaged = new MongoManaged(configuration.mongo); environment.lifecycle().manage(mongoManaged); environment.healthChecks().register( "MongoHealthCheck", new MongoHealthCheck(mongoMa environment.jersey().register(new SubscriptionResour environment.jersey().register(new BillResource(mongo } DeployDeploy
  22. 22. 1. Install Java (defaults to OpenJDK) 2. Open firewall ports 8080 and 8081 3. Copy file to server 4. Run fatjar sudo apt-get install java7-runtime-headless gcutil addfirewall rest --description="http" --allowed gcutil addfirewall admin --description="Iadmin" --allo gcutil --project={project-id} push {instance-name} {lo java -jar billproto-0.2-fat.jar server billproto.yml
  23. 23. TestingTesting REST urlsREST urls Subscription: Trigger: Bill: Healthcheck: 162.222.183.244:8080/subscription/ 162.222.183.244:8080/subscription/{subscriptionID}/{billID} 162.222.183.244:8080/bill/?billID={billID} 162.222.183.244:8081
  24. 24. ConclusionConclusion By building just the bare minimum operations in each components that work well together, we were able to cover the core user story and better assess our API feasibility. We also managed to test out a new framework and built a template that can be fleshed out for future applications.
  25. 25. Links and CreditsLinks and Credits built with dropwizard.io mongojack.org gradle.org This project source code in GitHub This Slide reveal.js Export to PDF
  26. 26. QuestionsQuestions

×