Building Your First Java Application with MongoDB

2,689 views
2,542 views

Published on

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,689
On SlideShare
0
From Embeds
0
Number of Embeds
1,279
Actions
Shares
0
Downloads
51
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Building Your First Java Application with MongoDB

  1. 1. Building your first Java app with MongoDB
  2. 2. Today’s webinar• Introduction to MongoDB.• Use a location-based surf report app as an example.• Basic data modelling, queries (inc geospatial) and aggregation.• MongoDB JS shell and Java code examples.
  3. 3. What is MongoDB?• MongoDB is a scalable, high-performance, open source NoSQL database. • Document-oriented storage • Full index support - including geospatial • Querying • Fast in-place updates • Map-Reduce and Aggregation • Replication and high availability • Horizontal scaling with auto-sharding • GridFS
  4. 4. Open source• MongoDB is an open source project• On GitHub• Licensed under the AGPL• Started & sponsored by 10gen• Commercial licenses available• Contributions welcome
  5. 5. High performance• Written in C++• Extensive use of memory-mapped files i.e. read-through write-through memory caching.• Runs nearly everywhere• Data serialized as BSON (fast parsing)• Full support for primary & secondary indexes• Document model = less work
  6. 6. Terminology
  7. 7. Example document{ "_id" :ObjectId("4c4ba5c0672c685e5e8aabf3"), "reporter" : "Hergé", "date" : ISODate("2012-02-02T11:52:27.442Z"), "location" : { "name" : "Watergate Bay", "state" : "Cornwall", "country" : "UK”}
  8. 8. Let’s build a location-based surf reporting app!
  9. 9. Let’s build a location-based surf reporting app!• Report current conditions
  10. 10. Let’s build a location-based surf reporting app!• Report current conditions• Get current local conditions
  11. 11. Let’s build a location-based surf reporting app!• Report current conditions• Get current local conditions• Determine best conditions per beach
  12. 12. Document structure{ "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "matt", "location" : { "coordinates" : [ -50.46667, 5.05 ], "name" : "Watergate Bay", "county" : "Cornwall" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 }, "date" : ISODate("2011-11-16T20:17:17.277Z")}
  13. 13. Document structure{ "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "matt", Unique, "location" : { "coordinates" : [ Auto-indexed -50.46667, 5.05 ], "name" : "Watergate Bay", "county" : "Cornwall" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 }, "date" : ISODate("2011-11-16T20:17:17.277Z")}
  14. 14. Document structure{ "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "matt", Unique, "location" : { "coordinates" : [ Auto-indexed -50.46667, 5.05 ], Geospatial "name" : "Watergate Bay", Compound Index "county" : "Cornwall" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 }, "date" : ISODate("2011-11-16T20:17:17.277Z")}
  15. 15. Document structure{ "_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key, "reporter" : "matt", Unique, "location" : { "coordinates" : [ Auto-indexed -50.46667, 5.05 ], Geospatial "name" : "Watergate Bay", Compound Index "county" : "Cornwall" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 Indexed for }, Time-To-Live "date" : ISODate("2011-11-16T20:17:17.277Z")}
  16. 16. Java: DBObjectsBasicDBObject report = new BasicDBObject();report.put("reporter", "matt");report.put("date", new Date());BasicDBObject location = new BasicDBObject();location.put("name", "Watergate Bay");location.put("county", "Cornwall");location.put("country", "UK");location.put("coordinates", new double[] {-50.46667, 5.05});report .put("location", location);BasicDBObject conditions = new BasicDBObject();conditions .put("height", 3);conditions .put("period", 13);conditions.put("rating", 5);report .put("conditions", conditions);
  17. 17. Connecting to MongoDBMongo m = new Mongo(Arrays.asList( new ServerAddress("localhost", 27017), new ServerAddress("localhost", 27018), new ServerAddress("localhost", 27019)));DB db = m.getDB( ”SurfReports” );DBCollection coll = db. getCollection(“reports”);
  18. 18. Inserting and finding a documentcoll.insert(report);DBObject report2 = coll.findOne();System.out.println(report2);{ "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "matt”, "location" : {"coordinates" : [-50.46667, 5.05], "name" : "Watergate Bay”, "county" : "Cornwall" },"conditions" : { "height" : 3, "period" : 13, "rating" : 5}, "date" : ISODate("2011-11-16T20:17:17.277Z”)}
  19. 19. Get local surf conditions - shell> db.reports.find( { "location.coordinates" : { $near : [50.41, -5.08] , $maxDistance : 0.5}, date : { $gte : new Date(2012, 11, 21)} }, {"location.name" :1, _id : 0, "conditions" :1}).sort({"conditions.rating" : -1})
  20. 20. Get local surf conditions - shell> db.reports.find( { "location.coordinates" : { $near : [50.41, -5.08] , $maxDistance : 0.5}, date : { $gte : new Date(2012, 11, 21)} }, {"location.name" :1, _id : 0, "conditions" :1}).sort({"conditions.rating" : -1}) • Get local reports
  21. 21. Get local surf conditions - shell> db.reports.find( { "location.coordinates" : { $near : [50.41, -5.08] , $maxDistance : 0.5}, date : { $gte : new Date(2012, 11, 21)} }, {"location.name" :1, _id : 0, "conditions" :1}).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports
  22. 22. Get local surf conditions - shell> db.reports.find( { "location.coordinates" : { $near : [50.41, -5.08] , $maxDistance : 0.5}, date : { $gte : new Date(2012, 11, 21)} }, {"location.name" :1, _id : 0, "conditions" :1}).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info
  23. 23. Get local surf conditions - shell> db.reports.find( { "location.coordinates" : { $near : [50.41, -5.08] , $maxDistance : 0.5}, date : { $gte : new Date(2012, 11, 21)} }, {"location.name" :1, _id : 0, "conditions" :1}).sort({"conditions.rating" : -1}) • Get local reports • Get today’s reports • Return only the relevant info • Show me the best surf first
  24. 24. Java: Building the query
  25. 25. Local surf conditions results{ "location" : { "name" : ”Watergate Bay" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 } }{ "location" : { "name" : "Watergate Bay" }, "conditions" : { "height" : 2.5, "period" : 9, "rating" :4 } }{ "location" : { "name" : ”Fistral North" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } }{ "location" : { "name" : "Fistral North" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } }{ "location" : { "name" : "Fistral North" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } }{ "location" : { "name" : ”Little Fistral" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } }{ "location" : { "name" : "Little Fistral" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } }{ "location" : { "name" : ”Cribbar" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } }{ "location" : { "name" : "Cribbar" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } }{ "location" : { "name" : "Cribbar" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } }
  26. 26. Analysis feature: Aggregation framework What are the best conditions for my beach?
  27. 27. Aggregation framework• Process a stream of documents • Original input is a collection • Final output is a result document• Series of operators • Filter or transform data • Input/output chainps ax | grep mongod | head -n 1
  28. 28. Pipelining operations $match Match “Fistral North” $project Only interested in conditions Group by rating, average $group wave height and wave period $sort Order by best conditions
  29. 29. Aggregation framework{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : ”Fistral North"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]}
  30. 30. Aggregation framework{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : ”Fistral North"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]} Match “Fistral North”
  31. 31. Aggregation framework{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : ”Fistral North"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]} Only interested in conditions
  32. 32. Aggregation framework{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : ”Fistral North"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]} Group by rating and average conditions
  33. 33. Aggregation framework{ "aggregate" : "reports" , "pipeline" : [ { "$match" : { "location.name" : ”Fistral North"}} , { "$project" : { "conditions" : 1}} , { "$group" : { "_id" : "$conditions.rating" , "average height" : { "$avg" : "$conditions.height"} , "average period" : { "$avg" : "$conditions.period"}}} , { "$sort" : { "_id" : -1}} ]} Show the best conditions first
  34. 34. Java: Aggregation helper
  35. 35. High availability: Replica sets• Initialize -> Election• Primary + data replication from primary to secondary Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary Replication Replication
  36. 36. Replica Set - failure• Primary down/network failure• Automatic election of new primary if majority exists Primary Election Node 1 Node 2 Secondary Heartbeat Secondary Node 3 Primary
  37. 37. Replica Set - failover• New primary elected• Replication established from new primary Node 1 Node 2 Secondary Heartbeat Primary Node 3 Primary
  38. 38. Durability• WriteConcern.SAFE• WriteConcern.JOURNAL_SAFE• WriteConcern.FSYNC_SAFE• WriteConcern.REPLICAS_SAFE• WriteConcern.MAJORITY• WriteConcern.NORMAL• WriteConcern.NONEm.setWriteConcern(WriteConcern.SAFE);coll.save(dbObj,WriteConcern.SAFE);
  39. 39. Read preferences• ReadPreference.primary()• ReadPreference.primaryPreferred()• ReadPreference.secondary()• ReadPreference.secondaryPreferred()• ReadPreference.nearest()ReadPreference preference = ReadPreference.primaryPreferred();DBCursor cur = new DBCursor(collection, query, null, preference);
  40. 40. Sharding• Automatic partitioning and management• Range based• Convert to sharded system with no downtime• Fully consistent• No code changes required
  41. 41. Scaling MongoDB MongoDB Single Instance or Replica Set Client Application
  42. 42. Scaling MongoDB Client Application
  43. 43. Mechanism of sharding Complete data setDefine shard key on location name Cribbar Fistral Little Sennen Watergate North Fistral Bay
  44. 44. Mechanism of sharding Chunk ChunkDefine shard key on location name Cribbar Fistral Little Sennen Watergate North Fistral Bay
  45. 45. Mechanism of sharding Chunk Chunk Chunk Chunk Cribbar Fistral Little Sennen Watergate North Fistral Bay
  46. 46. Mechanism of sharding Chunk Chunk Chunk Chunk Shard 1 Shard 2 Shard 3 Shard 4
  47. 47. Mechanism of sharding Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4
  48. 48. Mechanism of sharding Query: Watergate Bay Client Application config config config mongos Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 48
  49. 49. Mechanism of sharding Query: Cribbar Client Application config config config mongos Chu Chu nkc nkc Chu Chu Chu Chu Chu Chu Chu Chu nkc nkc nkc nkc nkc nkc nkc nkc Shard 1 Shard 2 Shard 3 Shard 4 49
  50. 50. More informationResource LocationMongoDB Downloads www.mongodb.org/downloadFree Online Training education.10gen.comWebinars and Events www.10gen.com/eventsWhite Papers www.10gen.com/white-papersCustomer Case Studies www.10gen.com/customersPresentations www.10gen.com/presentationsDocumentation docs.mongodb.orgAdditional Info info@10gen.com

×