Quick introduction to MongoDB in general and sharded cluster

1. Using MongoDB for BigData
May, 2017
... In 20 minutes

2. Agenda
• What is MongoDB?
• Replica Sets
• Sharding and Sharded Cluster
• MapReduce
• Spark and MongoDB

3. What is MongoDB?
• Open-source
• NoSQL
• Community / Enterprise versions
• Developed by MongoDB Inc. (formerly 10gen) in C++, C and JavaScript
• Cross-platform: Windows, Linux, OS X, Solaris, FreeBSD
• Document-oriented: stores extended binary JSON = BSON documents
• Stores any binary data like videos, pictures ... in GridFS
• Database development in JavaScript (standard libraries and user defined functions)
• Deploy, monitor, back up and scale MongoDB: Ops Manager
• Use MongoDB as a data source for your SQL-based BI: MongoDB Connector for BI,
SlamData
• Cross-platform UI for development: Robomongo
• Hosted MongoDB as a service: MongoDB Atlas
• Hosted platform for managing MongoDB: MongoDB Cloud Manager
• An another cloud provider: mLab

4. Some Organizations Rely on MongoDB

5. DB-Engines.com Top 10
Most popular NoSQL database

6. Terminology
RDBMS MongoDB
Database Database
Table Collection
Row Document
Index Index
Join Embedding
Partition Shard
Partition Key Shard Key

7. use blogdb
db.blog.insert({
"title" : "My Blog Post",
"content" : "Here's my blog post.",
"date" : ISODate("2016-08-24T21:12:09.982Z")
});
db.blog.find()
{ "_id" : ObjectId("591be0f4c79fa21e08c2e24e"), "title" : "My Blog Post", "content" :
"Here's my blog post.", "date" : ISODate("2016-08-24T21:12:09.982Z") }
Some Examples
collection object
method
current database object
generated universally unique primary key

8. db.blog.update(
{"_id" : ObjectId("591be0f4c79fa21e08c2e24e")},
{$push:
{"comments":
{"user":"usr1", "date":ISODate("2016-08-24T22:12:09.982Z"), text:"first comment"}
}})
db.blog.find().pretty()
{
"_id" : ObjectId("591be0f4c79fa21e08c2e24e"),
"title" : "My Blog Post",
"content" : "Here's my blog post.",
"date" : ISODate("2016-08-24T21:12:09.982Z"),
"comments" : [
{
"user" : "usr1",
"date" : ISODate("2016-08-24T22:12:09.982Z"),
"text" : "first comment"
}
]
}
Some Examples
embedding

9. orders collection:
MapReduce Example
{
_id: ObjectId("50a8240b927d5d8b5891743c"),
cust_id: "abc123",
ord_date: new Date("Oct 04, 2012"),
status: 'A',
price: 25,
items: [ { sku: "mmm", qty: 5, price: 2.5 },
{ sku: "nnn", qty: 5, price: 2.5 } ]
}
var mapFunction1 = function() {
emit(this.cust_id, this.price);
};
var reduceFunction1 = function(keyCustId, valuesPrices) {
return Array.sum(valuesPrices);
};
db.orders.mapReduce(
mapFunction1,
reduceFunction1,
{ out: "map_reduce_example" }
)

10. Replica Sets: high availability
Read (optional) Read (optional)
Replication

11. Replica sets: automatic failover

12. Sharded Cluster
Shards are replica sets

13. Sharding
Ranged
Hashed
Zone
Manually associate shard key ranges to zones (groups of shards)

14. MongoDB connector to Apache Spark
Can be sharded
clusters too!
Data can be filtered,
aggregated at MongoDB
level

15. • Speedy
• Highly available
• Flexible data model
• Simple to use
• Infinite data size
BUT
• Sharded Cluster deployment requires planning!
Summary

16. • Install a MongoDB server / sign up to a free hosted MongoDB service like mLab sandbox
• Load the postcodes.zip data file using the mongoimport utility. If you use a MongoDB service, you will
need to install MongoDB client on your machine first.
• Create a Btree index on place.name, postal_code, place.name + place.country and
place.country fields
• Create a 2dsphere index on place.loc
• Add the {"postal_code" : "38116", "place" : { "name" : "Graceland", "country"
:"US", "state" : "Memphis", "loc" : [ 19.0419, 47.5328 ] } } document to the
collection
• Change the place.loc field of the same document to [-90.02604930000001, 35.0476912]
• Add the field owner: Lisa Marie Presley to the same document. Observe that the structure of
the document is different from the other documents of the collection.
Send me the queries that answer to the following questions:
• What is the value of the postal code of Graceland/Memphis? We need only the {"postal_code" :
"38116"} document, fields other than postal_code are not acceptable!
• How many postal_codes are in Budapest/Hungary?
• When was the "59199cdff0269ea12235e9dc" ObjectId created?
• Top 5 countries by number of documents in descending order
• Which places are within 20km around longitude -90.02604930000001 and latitude 35.0476912
(Graceland)? The result must be sorted in alphabetical order and each place appear in the result only
once (distinct).
Homework

17. Questions
?

18. Thank You!