SlideShare a Scribd company logo
1 of 58
INTRODUCING NOSQL
AND
ONGODB
Pom Bleeksma & Lucas Jellema
14 March 2017, Nieuwegein
M
AGENDA
INTRODUCTION WHAT’S NOT SQL ABOUT
NOSQL – HISTORY,
BACKGROUND, USE CASES
OVERVIEW OF NOSQL
DATABASES
INTRODUCING MONGODB GETTING STARTED WITH
MONGODB – INSTALLATION,
CONFIGURATION,
ADMINISTRATION
DEVELOPING WITH MONGODB
– JAVA & NODE.JS
HANDSON WORKSHOP
TRADITIONAL APPROACH:
ALL DATA IN ENTERPRISE RELATIONAL DATABASE
TRADITIONAL APPROACH:
WITHOUT MUCH REGARD FOR
• What is it used for?
• By whom, where, in what way, using which tools
• Ratio between writes and reads
• Ad hoc access?
• What format is it in/should it be in?
• What confidentiality & integrity is required?
• How much of it?
• How long relevant (hot vs cold vs dead)?
• How fine grained and how accurate?
• How much change?
DATABASE: SINGLE POINT OF …
SCALABILITY | AVAILABILITY | CONSISTENCY
• Single Point of …
• What goes where …
RELATIONAL DATABASES
• Based on relational model of data (E.F. Codd), a mathematical
foundation
• Uses SQL for query, DML and DDL
• Transactions are ACID (Atomicity, Consistency, Isolation, Durability)
• All or nothing
• Constraint Compliant
• Individual experience [in a multi-session environment]
(aka concurrency)
• Down does not hurt
6
THE HOLY GRAIL OF NORMALIZATION
• Normalize to prevent data redundancy, discrepancies (split brain) and
storage waste
• Pragmatic approaches - recognizing the fact that some data is read far
more frequently than that it is created and modified
• Materialized View
• Virtual Column
• Index
• Sharding
• Replication
• Data Guard
• Data Warehouse & Data Marts
• Client Side Caching
• In Memory Database
ACID COMES AT A COST
• Transaction results have to be persisted in order to guarantee D
• Concurrency requires some degree of locking (and multi-versioning) in order to have I
• Constraint compliance (unique key, foreign key) means all data hangs together (as do all
transactions) in order to have C
8
ACID COMES AT A COST
• ACID means:
• All data closely together (and sometimes far from
applications and users)
• All transactions centrally organized
• All data persisted as part of the transaction
• ACID means:
• Hard to scale out (horizontally)
• Oracle offers scale up with Real Application Clusters
9
THE RELATIONAL MODEL IN PRACTICE
• Traditional Relational Data Model has severe impact on physical disk
performance
• Transaction Log => Sequential Write (append to file)
• Data Blocks require much more expensive Random Access disk writes
• Indexes (B-Tree, Bitmap, …) are used to speed up query (read) performance
• and slow down transactions
• Relational data does not [always] map naturally to data format required in
application (OO, JSON, XML)
• Capability to join and construct ad-hoc queries across the entire data model is
powerful
• Declarative integrity constraints allow for strict enforcement of data quality rules
• “the data may be non sensical, but at least it adheres to the rules, whatever the origin”
• SQL has been embraced by huge numbers of tools & technologies and IT/Data
professionals
DATABASES RE-EVALUATED
• Not all use cases require ACID (or can afford it)
• Read only (product catalog for web shops)
• Inserts only and no (inter-record) constraints
• Big Data collected and “dumped” in Data Lake (Hadoop) for subsequent
processing
• High performance demands
• Not all data needs structured formats or structured querying and JOINs
• Entire documents are stored and retrieved based on a single key
• Sometimes – scalable availability is more important than Consistency – and
ACID is sacrificed
• CAP-theorem states: Consistency [across nodes], Availability and Partition
tolerance can not all three be satisfied
11
USE CASE: WEBSHOP
• Webshop – 1M visitors per day
• Product catalog consists of millions of records
• The web shop presents: product description, images, reviews, pricing
details, related offerings, stock status
• Products are added and updated and removed every day
• Although most products do not change very frequently
• Some vendors do bulk manipulation of product details
Products Product updates
Webshop visits
- searches
- product details
- orders
USE CASE: WEBSHOP
TECHNOLOGY AND ARCHITECTURE
CONSIDERATIONS
Products Product updates
Webshop visits
- searches
- product details
- orders
firewall
Data manipulation
Data Quality (enforcement)
<10K transactions
Batch jobs next to online
Speed is nice
Read only
On line
Speed is crucial
XHTML & JSON
> 1M visits
Products
Products
USE CASE: WEBSHOP
TECHNOLOGY AND ARCHITECTURE CONSIDERATIONS
Products
Webshop visits
- searches
- product details
- orders
firewall
Data manipulation
Data Quality (enforcement)
<10K transactions
Batch jobs next to online
Speed is nice
Read only
On line
Speed is crucial
XHTML & JSON
> 1M visits
DMZ
Read only
JSON documents
Images
Text Search
Scale Horizontally
Products
Nightly generation
Product updates
A FEW OBSERVATIONS
• Redundancy
• for performance, scalability, availability
• Commodity hardware for scale out architecture
• Data (documents) prepared for specific application usage (super MV)
• Format
• Search paths
• Time lag in data modifications is accepted
• Perhaps even desirable to offer a consistent view to webshop users
• Transactions in one engine
• ACID still in place
• Transactions do not compete with reads
• vying for the same physical resources
NOSQL AND BASE
• NoSQL arose because of performance and scalability
challenges with Web Scale operations
• NoSQL is a label for a wide variety of databases that lack some
aspect of a true relational database
• ACID-ness, SQL, relational model, constraints
• The label has been used since 2009
• Perhaps NoREL would be more appropriate
• Some well known NoSQL products are
• Cassandra, MongoDB, Redis, CouchDB, …
• BASE as alternative to ACID:
• basically available, soft state, eventually consistent
(after a short duration)
16
TYPICAL FOR NOSQL
• Focus on speed, availability and scalability
• Horizontal scale out – distributed with load balancing and fail-over
• No (predefined) Data Structure
• Integrity primarily protected by application logic
• Open Source (most offerings are, not all: MarkLogic)
• Close(r) attention for how the data is used
• Application oriented data format and search paths and specialized database per
application (microservice, capability)
• Similar to the switch from SOA to API/Microservice
• Reads (far) more relevant than writes
• Data redundancy & denormalization
• No data access through SQL
NOSQL MEANS:
NO DATA ACCESS THROUGH SQL
• However
• Data Professionals and
Developers speak SQL
• Reporting, Dashboarding,
ETL, BI tools speak SQL
• There is no common query
language across NoSQL
products
NO DATA ACCESS THROUGH SQL
• However
• Data Professionals and
Developers speak SQL
• Reporting, Dashboarding,
ETL, BI tools speak SQL
• There is no common query
language across NoSQL
products
• Attempts from many vendors to create drivers that translate SQL
statements into NoSQL commands for the specific target database
• To protect existing investments in SQL – skills, tools, applications, reports, ..
TYPES OF NOSQL DATABASES
• Key-Value (from in-memory cache to hardware appliance –
Memcached, Hazelcast, Coherence, Redis, Oracle NoSQL/Berkeley
DB)
• Wide Column Store (Cassandra, Google BigTable, HBase)
• Document (JSON, XML, YAML and binary forms – MongoDB,
CouchDB, Couchbase, MarkLogic)
• Graph (networks of data, triplets, RDF stores – Neo4J)
• Miscellaneous: object, tabular, tuple, elastic search index, Hadoop
(hdfs) style, Kafka Topic
(LEADING) NOSQL DATABASE PRODUCTS
• MongoDB is (one of) the most popular (by any measure)
• Cloud (only):
• Google BigTable,
• AWS Dynamo
• Cache (in memory)
• ZooKeeper, Redis,
Memcached, …
• Hadoop/HDFS
• Oracle NoSQL
(fka Berkeley DB)
HISTORY OF MONGODB
• 10gen – startup from 2007 in New York City
• Developed database as component in Platform as a Service product
• Initial release of MongoDB: 2009
• 10gen changed its name to MongoDB Inc. in 2013
• Offers enterprise support, MongoDB Atlas cloud service, MongoDB Compass, Connectors
• Current stable release: 3.4.2 (3.5.2 is in preview)
• Open Source - GNU Affero General Public License and the Apache License.
• MongoDB == Humongous Database
• Evolution of MongoDB is still pretty fast
• Some crucial enterprise database capabilities were added fairly recently or are still lacking
• MongoDB is number one NoSQL database in terms of popularity
CORE ARCHITECTURE
• MongoDB stores JSON documents in a binary format (BSON)
• Documents are stored in collections (similar to row or records in tables)
• Interaction is through JavaScript
• taking the place of SQL for DML and query
• Written in C++
• Has the V8 JavaScript engine included
• Runs on Mac OS X, Windows, Solaris, and most flavors of Linux
• Source code is on GitHub
• Has GeoSpatial and Tekst indexes and search capabilities
• MongoDB, Inc. officially supports drivers for C, C++, C#, Erlang, Java, Node.js,
JavaScript, Perl, PHP, Python, Scala, and Ruby
ARCHITECTUUR, INSTALLATIE,
CONFIGURATIE, OPSLAG, SHELL
• Mongo draait op windows en linux.
• Windows install met msi file
• linux install met tgz file.
• Users en directories vrij te kiezen
• Mongod --dbpath db1-- port 27001
• Default path=datadb, default port=27017
• Mongo [-- port hostname:27001] [--shell some_json_file]
• Dbs, collections
25
IMPORT EXPORT
• Mongoimport
• Json
• Csv, tsv, maar hebben een descriptor file nodig
• mongoexport -h -d -c -o -q
• Leesbare output, minder geschikt voor backups
CRUD
• show dbs
• use db
• db.<collection>.find()
• db.<collection>.find().pretty()
• select: db.find({key:value,key:value})
• present: db.<collection>.find({key:value},{key:1,key:1,_id:0})
• explain: db.find({key:value,key:value}).explain("executionStats")
• db.companies.insert({Company:"AMIS",Event:"MongoDB Workshop"})
• db.companies.update({Company:"AMIS"},{$set: {Location:"Nieuwegein"}})
• Geen commit! Ofwel, autocommit.
CRUD
• db.companies.remove({"Company" : "AMIS"})
• $unset -> Om een veld uit een document te verwijderen.
• $inc -> om een numeriek veld te verhogen met een bepaalde waarde. Als het
veld niet bestaat wordt het toegevoegd (met de waarde van de verhoging)
• $push -> Om een waarde aan een array toe te voegen.
INDEXING, EXPLAIN
• _Id altijd geïndexeerd en unique
• Indexen helpen bij zoeken
• Indexen mogen compound zijn, dwz op meerdere keys gecombineerd, bv
index op woonplaats+leeftijd
• db.user.find().sort({ woonplaats:1, leeftijd: -1 }) De -1 betekent aflopend
geïndexeerd.
• Resultaten rechtstreeks uit index indien mogelijk (fat indexes in SQL)
• db.user.find({woonplaats:”Zoetermeer”}).explain(“executionStats”) geeft een
goed overzicht van het gevolgde plan
REPLICA SETS
• Replica sets zijn gedupliceerde collections
• Beveilliging tegen uitval
• De members besluiten onderling welk member primary is. Alleen
daarop kan DML plaatsvinden
• Altijd oneven aantal members, eventueel mbv een arbiter
• Secondary members kunnen wel voor select gebruikt worden, als je
eerst slaveOk() ingeeft
• Mogen op dezelfde host draaien, mogen ook op verschillende hosts
draaien.
• writeConcern: Je kan expliciet aangeven op hoeveel members je
DML uitgevoerd moet zijn voor je je prompt terug krijgt.
SHARDING
• Shard = splinter
• Range based partitioning
• Altijd index nodig op de shard key. Hoeft niet unique te zijn.
• Iedere shard is een proces. Bij voorkeur ieder proces zijn eigen host.
• Scale out proces
• Metadata staat in aparte config db. Typically 3 stuks.
• Clients verbinden met een mongos server. Dat is een instance
zonder database die uit de config haalt waar de data zich bevindt.
• Iedere shard kan bestaan uit replica sets.
SHARDING
• Data verdeelt zichzelf over chunks.
• Chunks kunnen splitten wanneer ze te groot worden (typically
100MB maar is aan te passen naar wens)
• Chunks kunnen migraten naar een andere chard. Mongo streeft
ernaar elke shard evenveel chunks te laten hebben.
• Tijdens migration kan DML gewoon doorgaan.
BACKUP & RESTORE
• Backup met mongodump
• Creëert een dump directory met daaronder een directory per
database en plaatst daarin 1 of meerdere .bson files, een binair
formaat.
• Kan vanaf ieder member van een replica set. Bij voorkeur eentje die
het niet druk heeft.
• Mongo kent geen transactions, dus backup is niet consistent. Het is
maar afwachten wat erin staat als er veel DML plaatsvindt.
• De tegenhanger heet mongorestore.
• Restore van bestaande database lukt niet: duplicate key op _id
MONGODB COMPASS
(AKA NOSQL DEVELOPER)
• The GUI for MongoDB
• Visually explore data
• Run ad hoc queries in seconds.
• Interact with your data with full CRUD functionality
• View and optimize your query performance.
• Separately installed product
• On Windows, OS X, Linux, …
• Can run against local and remote MongoDB servers
MONGODB COMPASS
PROGRAMMING AGAINST MONGODB
• MongoDB officially supports drivers for C, C++, C#, Erlang, Java,
Node.js, JavaScript, Perl, PHP, Python, Scala, and Ruby
• Note: there is also a Connector for Hadoop
• Several REST APIs for MongoDB are available from the community
• RESTHeart (Java), Eve (Python),, Crest (Node), AMID, Kule, DreamFactory
Connect,
create,
update
Connect,
create,
update
MEAN STACK
• Modelled after LAMP
• End-to-End JavaScript/JSON
• Term coined in 2013 (by MongoDB)
INTERACTING WITH MONGODB
FROM NODE.JS
• MongoDB Node.js Driver 2.0
• Support for ECMAScript 6.0 – Promises for asynch
• http://mongodb.github.io/node-mongodb-native/2.0/
• Using mongodb in a NodeJS application
• npm install mongodb –save
• var MongoClient = require('mongodb').MongoClient;
Connect,
create,
update
CONNECTING TO SERVER
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
MongoClient.connect(url, function(err, db) {
console.log("Connected correctly to server.");
// DO YOUR THING WITH MONGODB
db.close();
console.log("Connection to database is closed.");
}) //connect()
RETRIEVE DATA (1)
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
MongoClient.connect(url, function(err, db) {
console.log("Connected correctly to server.");
db.collection('countries').find({},{"sort": [["area",-1]]})
.limit(20).toArray(function(err, results){
console.log("Name of Country Four " +results[3].name+ " and size: " +results[3].area);
// using cursors to retrieve data sets in a controlled fashion
// note: cursor implements NodeJS Stream - results can be piped
var cursor = db.collection('countries').find({"continent":"Asia"}, {"sort": "name"});
cursor.count(function(err, count){
console.log("Country count in Asia: "+count);
});
cursor.each(function(err, country){
console.log(country.name);
})//each cursor
RETRIEVE DATA (2)
Connect,
create,
update
var aggquery = [ {$sort: {area : -1}}
, {$group:{ _id: '$continent'
, largestCountry : {$first: "$name"}
}
}
];
var aggcursor = db.collection('countries').aggregate(aggquery);
aggcursor.each(function(err, result){
if (err) {
console.log(err);
} else if (result)
console.log(JSON.stringify(result));
}) //each aggcursor
var ccursor = db.collection('countries').find({});
// the cursor returned from find and aggregate implements a NodeJS (readable) Stream
ccursor.on('data', function(doc) {
console.log(doc);
});
ccursor.once('end', function() {
console.log(“Out of countries. Time to move on");
});
CONNECTING TO SERVER – ECMA 6
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var assert = require('assert'), co = require('co');
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
co(function*() {
// Use connect method to connect to the Server
var db = yield MongoClient.connect(url);
console.log("Connected correctly to server");
// DO YOUR THING WITH MONGODB
// Close the connection
db.close();
}).catch(function(err) {
console.log(err.stack);
});
RETRIEVE DOCUMENTS – ECMA 6
Connect,
create,
update
co(function*() {
// Use connect method to connect to the Server
var db = yield MongoClient.connect(url);
console.log("Connected correctly to server");
// find top 20 countries by size
var results = yield db.collection('countries').find({}
,{"sort": [["area",-1]]}).limit(20).toArray();
console.log("Country One " +JSON.stringify(results[0]));
console.log("Name of Country Four " +results[3].name
+ " and size: " +results[3].area);
// use cursor to get the country count
var cursor = db.collection('countries').find({"continent":"Asia"}
, {"sort": "name"});
var count = yield cursor.count();
console.log("Country count in Asia: "+count);
while (yield cursor.hasNext()){
var country = yield cursor.next();
console.log(country.name);
}
RETRIEVE DOCUMENTS (2) – ECMA 6
Connect,
create,
update
// the largest country per continent
var aggquery = [ {$sort: {area : -1}}
, {$group:{ _id: '$continent'
, largestCountry : {$first: "$name"}
}
}
];
var aggcursor = db.collection('countries').aggregate(aggquery);
while (yield aggcursor.hasNext()){
var result = yield aggcursor.next();
console.log(JSON.stringify(result));
}
// Close the connection
db.close();
}).catch(function(err) {
console.log(err.stack);
});//co
CREATE DOCS IN MONGODB – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {
"continent" : "Europe",
"children" : [ {"name":"Belgium"}, {"name":"Luxemburg"}],
"someVariable" : 19123, "andmore" : "2716NK" };
var doc2 = {
"continent" : "Asia", "name" : "Johnny",
"nrs" : [ {"name":"China"}, {"name":"India"}, {"name":"Buthan"}],
"tree": {"branch": {"twig":{"leaf": 1}}}
};
var nameOfCollection = "myDocs"
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount
+" documents created into collection "+nameOfCollection);
...
CREATE & UPDATE DOCS – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“;
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount +" documents created);
var cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
result = yield db.collection(nameOfCollection).updateOne(
{"tree.branch.twig.leaf":1}, {$set: {name: "Hank", city:"Nieuwegein“
, "location.province":"Utrecht“
, "location.country":"The Netherlands“
, "tree.stem":5}});
console.log(">> Updated "+result.modifiedCount+" document(s)");
cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
CREATE, UPDATE AND DELETE – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“;
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount +" documents created);
var cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
result = yield db.collection(nameOfCollection).updateOne(...);
...
result = yield db.collection(nameOfCollection).deleteMany({});
console.log(">> Deleted "+ result.deletedCount+" documents ");
// execute command to drop the collection
yield db.command({drop:nameOfCollection});
console.log(">> Dropped collection "+nameOfCollection);
// Close the connection
db.close();
console.log("Connection to database is closed.");
INTERACTING WITH MONGODB
FROM JAVA
• MongoDB Java Driver 3.x
• Supports Synchronous and Asynchronous operations
• https://docs.mongodb.com/ecosystem/drivers/java/
• Using MongoDB Driver in a Java Application
• Add dependency in Maven pom.xml file
• Library JAR file mongo-java-driver-3.4.0.jar
Connect,
create,
update
CONNECTING TO SERVER
package nl.amis.mongodb.countries;
import java.net.UnknownHostException;
import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
String mongodbHost = "127.0.0.1";
Integer mongodbPort = 27017;
String mongodbDatabase = "world";
MongoClient mongoClient = new MongoClient(mongodbHost, mongodbPort);
try {
System.out.println("Connected to server; now hook into database " + mongodbDatabase);
MongoDatabase db = mongoClient.getDatabase(mongodbDatabase);
System.out.println("Connected to database " + mongodbDatabase + " successfully");
} finally {
mongoClient.close();
System.out.println("Closed mongodb connection");
}
}
}
Connect,
create,
update
RETRIEVE DATA
...
import org.bson.Document;
import com.mongodb.client.MongoCollection;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
... Connect to world database
// get all documents from the countries collection
MongoCollection<Document> countries = db.getCollection("countries");
System.out.println("Got collection countries successfully");
System.out.println("Number of countries: " + countries.count());
System.out.println("All countries - name and continent");
for (Document doc : countries.find()) {
// System.out.println(doc.toJson());
System.out.println(doc.get("name") + " (" + doc.get("continent") + ")");
}//for
Connect,
create,
update
RETRIEVE DATA (2)
import com.mongodb.client.model.Sorts;
import com.mongodb.client.model.Filters;
import com.mongodb.Block;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
... Connect to world database
System.out.println( "Retrieve all countries in Asia order alphabetically
by name and display full country document");
countries.find(Filters.eq("continent", "Asia"))
.sort(Sorts.ascending("name"))
.forEach(new Block<Document>() {
@Override
public void apply(final Document document) {
System.out.println(document.toJson());
}
});
Connect,
create,
update
RETRIEVE DATA - AGGREGATION
...
Block<Document> printBlock = new Block<Document>() {
@Override
public void apply(final Document document) {
System.out.println("Country: " + document.toJson());
}
};
...
System.out.println("List largest country in each continent");
countries.aggregate(
Arrays.asList(
Aggregates.sort(Sorts.descending("area")),
Aggregates.group("$continent", Accumulators.first("Largest Country", "$name"))
))
.forEach(printBlock);
Connect,
create,
update
RETRIEVE DATA – AGGREGATION (2)
System.out.println("List name in uppercase,sorted by size, delta between birthdate and deathrate, "
+" population density for countries in Oceania");
BsonArray subArgs=new BsonArray();
subArgs.add(new BsonString("$birthrate"));
subArgs.add(new BsonString("$deathrate"));
BsonArray divArgs=new BsonArray();
divArgs.add(new BsonString("$population"));
divArgs.add(new BsonString("$area"));
countries.aggregate
(Arrays.asList(
Aggregates.match(Filters.eq("continent", "Oceania")),
Aggregates.sort(Sorts.descending("area")),
Aggregates.project( Projections.fields(
Projections.excludeId(),
Projections.include("continent"),
Projections.computed("name",Projections.computed("$toUpper", "$name")),
Projections.computed("populationGrowthRate",Projections.computed("$subtract", subArgs)),
Projections.computed("populationDensity",
Projections.computed("$trunc", Projections.computed("$divide", divArgs)))
))
))
.forEach(printBlock);
Connect,
create,
update
JAVA: CREATE DOCUMENTS IN MONGODB
...
String nameOfCollection = "myDocs";
MongoCollection<Document> coll = db.getCollection(nameOfCollection);
Document doc1 = new Document("continent", "Europe")
.append("nrs", Arrays.asList(new Document("name", "Belgium"),
new Document("name", "Luxemburg")))
.append("someVariable", 19123)
.append("andmore", "kl;jdsfakhfsdahjfsdasjbsdahjbsdahgvsahjkZl;po");
Document doc2 = new Document("continent", "Asia").append("name", "Johnny")
.append("nrs", Arrays.asList(new Document("name", "China"),
new Document("name", "India"), new Document("name", "Buthan")))
.append("tree", new Document("branch",
new Document("twig", new Document("leaf", 1))));
List<Document> documents = new ArrayList<Document>();
documents.add(doc1);
documents.add(doc2);
coll.insertMany(documents);
System.out.println(">> Documents created into collection");
coll.find().forEach(printBlock);
Connect,
create,
update
JAVA: UPDATE AND DELETE DATA
...
coll.updateOne(
Filters.eq("tree.branch.twig.leaf", 1),
Updates.combine(
Updates.set("name", "Hank"), Updates.set("city", "Nieuwegein"),
Updates.set("location.province", "Utrecht"),
Updates.set("location.country", "The Netherlands"),
Updates.set("tree.stemp", 5),
Updates.currentDate("lastModified")
)
);
System.out.println(">> Updated one document in collection " + nameOfCollection);
coll.find().forEach(printBlock);
DeleteResult dr = coll.deleteMany(Filters.exists("_id"));
System.out.println(">> Deleted " + dr.getDeletedCount() + " documents from collection "
+ nameOfCollection);
db.runCommand(new Document("drop", nameOfCollection));
System.out.println(">> Dropped collection " + nameOfCollection);
Connect,
create,
update
INTERACTING WITH MONGODB
FROM JAVA – JSON  OO MAPPING
• Libraries for mapping between BSON/JSON
documents and Java Objects
• Morphia - http://mongodb.github.io/morphia/1.3/getting-started/quick-tour/
• Jongo – Query in Java as in the MongoDB shell - http://jongo.org/
Connect,
create,
update
HANDS ON WORKSHOP
• Install MongoDB
• Create Database, Create Collections and Documents
• Retrieve Data
• Import Database
• More configuration and administration
• Development: interacting with MongoDB from
• Compass
• Java
• Node.js
• Resources: https://github.com/lucasjellema/sig-nosql-mongodb
• Blog: technology.amis.nl
• Email: pom.bleeksma@amis.nl , lucas.jellema@amis.nl
• : @lucasjellema
• : lucas-jellema
• : www.amis.nl, info@amis.nl
+31 306016000
Edisonbaan 15,
Nieuwegein

More Related Content

What's hot

Responding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyResponding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyAlibaba Cloud
 
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsCloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsBlazeclan Technologies Private Limited
 
Building Complete Private Clouds with Apache CloudStack and Riak CS
Building Complete Private Clouds with Apache CloudStack and Riak CSBuilding Complete Private Clouds with Apache CloudStack and Riak CS
Building Complete Private Clouds with Apache CloudStack and Riak CSJohn Burwell
 
Logging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorLogging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorCask Data
 
Understanding Codenvy - for Containerized Developer Workspaces
Understanding Codenvy - for Containerized Developer WorkspacesUnderstanding Codenvy - for Containerized Developer Workspaces
Understanding Codenvy - for Containerized Developer WorkspacesLynn Langit
 
When the Cloud is a Rockin: High Availability in Apache CloudStack
When the Cloud is a Rockin: High Availability in Apache CloudStackWhen the Cloud is a Rockin: High Availability in Apache CloudStack
When the Cloud is a Rockin: High Availability in Apache CloudStackJohn Burwell
 
Designing For Multicloud, CF Summit Frankfurt 2016
Designing For Multicloud, CF Summit Frankfurt 2016Designing For Multicloud, CF Summit Frankfurt 2016
Designing For Multicloud, CF Summit Frankfurt 2016Mark D'Cunha
 
New Roles In The Cloud
New Roles In The CloudNew Roles In The Cloud
New Roles In The CloudSteve Loughran
 
RedisConf18 - Transforming Vulnerability Telemetry with Redis Enterprise
RedisConf18 - Transforming Vulnerability Telemetry with Redis EnterpriseRedisConf18 - Transforming Vulnerability Telemetry with Redis Enterprise
RedisConf18 - Transforming Vulnerability Telemetry with Redis EnterpriseRedis Labs
 
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017Codemotion
 
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig Narvaez
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig NarvaezServerless Architectures with AWS Lambda and MongoDB Atlas by Sig Narvaez
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig NarvaezData Con LA
 
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetApp
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetAppBridging Your Business Across the Enterprise and Cloud with MongoDB and NetApp
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetAppMongoDB
 
Discovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clustersDiscovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clustersIvan Donev
 
Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019Ivan Donev
 
Bi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stackBi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stackIvan Donev
 
Perth Azure Usergroup Build 2018 updates
Perth Azure Usergroup Build 2018 updatesPerth Azure Usergroup Build 2018 updates
Perth Azure Usergroup Build 2018 updatesNirmal Thewarathanthri
 
Azure Cosmos DB: Features, Practical Use and Optimization "
Azure Cosmos DB: Features, Practical Use and Optimization "Azure Cosmos DB: Features, Practical Use and Optimization "
Azure Cosmos DB: Features, Practical Use and Optimization "GlobalLogic Ukraine
 

What's hot (20)

Working and Features of HTML5 and PhoneGap - An Overview
Working and Features of HTML5 and PhoneGap - An OverviewWorking and Features of HTML5 and PhoneGap - An Overview
Working and Features of HTML5 and PhoneGap - An Overview
 
MongoDB 3.4 webinar
MongoDB 3.4 webinarMongoDB 3.4 webinar
MongoDB 3.4 webinar
 
Responding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyResponding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database Technology
 
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring ReportsCloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
 
Building Complete Private Clouds with Apache CloudStack and Riak CS
Building Complete Private Clouds with Apache CloudStack and Riak CSBuilding Complete Private Clouds with Apache CloudStack and Riak CS
Building Complete Private Clouds with Apache CloudStack and Riak CS
 
Logging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorLogging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data Collector
 
Understanding Codenvy - for Containerized Developer Workspaces
Understanding Codenvy - for Containerized Developer WorkspacesUnderstanding Codenvy - for Containerized Developer Workspaces
Understanding Codenvy - for Containerized Developer Workspaces
 
When the Cloud is a Rockin: High Availability in Apache CloudStack
When the Cloud is a Rockin: High Availability in Apache CloudStackWhen the Cloud is a Rockin: High Availability in Apache CloudStack
When the Cloud is a Rockin: High Availability in Apache CloudStack
 
Designing For Multicloud, CF Summit Frankfurt 2016
Designing For Multicloud, CF Summit Frankfurt 2016Designing For Multicloud, CF Summit Frankfurt 2016
Designing For Multicloud, CF Summit Frankfurt 2016
 
New Roles In The Cloud
New Roles In The CloudNew Roles In The Cloud
New Roles In The Cloud
 
RedisConf18 - Transforming Vulnerability Telemetry with Redis Enterprise
RedisConf18 - Transforming Vulnerability Telemetry with Redis EnterpriseRedisConf18 - Transforming Vulnerability Telemetry with Redis Enterprise
RedisConf18 - Transforming Vulnerability Telemetry with Redis Enterprise
 
Amazon Reshift as your Data Warehouse Solution
Amazon Reshift as your Data Warehouse SolutionAmazon Reshift as your Data Warehouse Solution
Amazon Reshift as your Data Warehouse Solution
 
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
 
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig Narvaez
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig NarvaezServerless Architectures with AWS Lambda and MongoDB Atlas by Sig Narvaez
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig Narvaez
 
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetApp
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetAppBridging Your Business Across the Enterprise and Cloud with MongoDB and NetApp
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetApp
 
Discovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clustersDiscovery Day 2019 Sofia - Big data clusters
Discovery Day 2019 Sofia - Big data clusters
 
Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019Discovery Day 2019 Sofia - What is new in SQL Server 2019
Discovery Day 2019 Sofia - What is new in SQL Server 2019
 
Bi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stackBi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stack
 
Perth Azure Usergroup Build 2018 updates
Perth Azure Usergroup Build 2018 updatesPerth Azure Usergroup Build 2018 updates
Perth Azure Usergroup Build 2018 updates
 
Azure Cosmos DB: Features, Practical Use and Optimization "
Azure Cosmos DB: Features, Practical Use and Optimization "Azure Cosmos DB: Features, Practical Use and Optimization "
Azure Cosmos DB: Features, Practical Use and Optimization "
 

Similar to Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14-03-2017)

When to Use MongoDB
When to Use MongoDBWhen to Use MongoDB
When to Use MongoDBMongoDB
 
Got documents - The Raven Bouns Edition
Got documents - The Raven Bouns EditionGot documents - The Raven Bouns Edition
Got documents - The Raven Bouns EditionMaggie Pint
 
Which postgres is_right_for_me_20130517
Which postgres is_right_for_me_20130517Which postgres is_right_for_me_20130517
Which postgres is_right_for_me_20130517EDB
 
Relational databases vs Non-relational databases
Relational databases vs Non-relational databasesRelational databases vs Non-relational databases
Relational databases vs Non-relational databasesJames Serra
 
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroli
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroliOptymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroli
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroliEDB
 
NoSQL and MongoDB Introdction
NoSQL and MongoDB IntrodctionNoSQL and MongoDB Introdction
NoSQL and MongoDB IntrodctionBrian Enochson
 
Optimize with Open Source
Optimize with Open SourceOptimize with Open Source
Optimize with Open SourceEDB
 
BPM und SOA machen mobil - Ein Architekturüberblick
BPM und SOA machen mobil - Ein ArchitekturüberblickBPM und SOA machen mobil - Ein Architekturüberblick
BPM und SOA machen mobil - Ein ArchitekturüberblickOPITZ CONSULTING Deutschland
 
BPM and SOA are going mobile - An architectural perspective
BPM and SOA are going mobile - An architectural perspectiveBPM and SOA are going mobile - An architectural perspective
BPM and SOA are going mobile - An architectural perspectiveOPITZ CONSULTING Deutschland
 
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesDropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesKyle Banerjee
 
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Lucidworks
 
Webinar: When to Use MongoDB
Webinar: When to Use MongoDBWebinar: When to Use MongoDB
Webinar: When to Use MongoDBMongoDB
 
Azure DocumentDB Overview
Azure DocumentDB OverviewAzure DocumentDB Overview
Azure DocumentDB OverviewAndrew Liu
 
Got documents Code Mash Revision
Got documents Code Mash RevisionGot documents Code Mash Revision
Got documents Code Mash RevisionMaggie Pint
 
EPAS + Cloud = Oracle Compatible Postgres in Minutes
EPAS + Cloud = Oracle Compatible Postgres in MinutesEPAS + Cloud = Oracle Compatible Postgres in Minutes
EPAS + Cloud = Oracle Compatible Postgres in MinutesEDB
 
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...GoQA
 

Similar to Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14-03-2017) (20)

When to Use MongoDB
When to Use MongoDBWhen to Use MongoDB
When to Use MongoDB
 
Oracle OpenWo2014 review part 03 three_paa_s_database
Oracle OpenWo2014 review part 03 three_paa_s_databaseOracle OpenWo2014 review part 03 three_paa_s_database
Oracle OpenWo2014 review part 03 three_paa_s_database
 
NoSQL
NoSQLNoSQL
NoSQL
 
Got documents - The Raven Bouns Edition
Got documents - The Raven Bouns EditionGot documents - The Raven Bouns Edition
Got documents - The Raven Bouns Edition
 
Which postgres is_right_for_me_20130517
Which postgres is_right_for_me_20130517Which postgres is_right_for_me_20130517
Which postgres is_right_for_me_20130517
 
Relational databases vs Non-relational databases
Relational databases vs Non-relational databasesRelational databases vs Non-relational databases
Relational databases vs Non-relational databases
 
CDC to the Max!
CDC to the Max!CDC to the Max!
CDC to the Max!
 
UNIT-2.pptx
UNIT-2.pptxUNIT-2.pptx
UNIT-2.pptx
 
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroli
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroliOptymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroli
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroli
 
NoSQL and MongoDB Introdction
NoSQL and MongoDB IntrodctionNoSQL and MongoDB Introdction
NoSQL and MongoDB Introdction
 
Optimize with Open Source
Optimize with Open SourceOptimize with Open Source
Optimize with Open Source
 
BPM und SOA machen mobil - Ein Architekturüberblick
BPM und SOA machen mobil - Ein ArchitekturüberblickBPM und SOA machen mobil - Ein Architekturüberblick
BPM und SOA machen mobil - Ein Architekturüberblick
 
BPM and SOA are going mobile - An architectural perspective
BPM and SOA are going mobile - An architectural perspectiveBPM and SOA are going mobile - An architectural perspective
BPM and SOA are going mobile - An architectural perspective
 
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesDropping ACID: Wrapping Your Mind Around NoSQL Databases
Dropping ACID: Wrapping Your Mind Around NoSQL Databases
 
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global
 
Webinar: When to Use MongoDB
Webinar: When to Use MongoDBWebinar: When to Use MongoDB
Webinar: When to Use MongoDB
 
Azure DocumentDB Overview
Azure DocumentDB OverviewAzure DocumentDB Overview
Azure DocumentDB Overview
 
Got documents Code Mash Revision
Got documents Code Mash RevisionGot documents Code Mash Revision
Got documents Code Mash Revision
 
EPAS + Cloud = Oracle Compatible Postgres in Minutes
EPAS + Cloud = Oracle Compatible Postgres in MinutesEPAS + Cloud = Oracle Compatible Postgres in Minutes
EPAS + Cloud = Oracle Compatible Postgres in Minutes
 
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...
 

More from Lucas Jellema

Introduction to web application development with Vue (for absolute beginners)...
Introduction to web application development with Vue (for absolute beginners)...Introduction to web application development with Vue (for absolute beginners)...
Introduction to web application development with Vue (for absolute beginners)...Lucas Jellema
 
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...Making the Shift Left - Bringing Ops to Dev before bringing applications to p...
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...Lucas Jellema
 
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...Lightweight coding in powerful Cloud Development Environments (DigitalXchange...
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...Lucas Jellema
 
Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema
 
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...Lucas Jellema
 
Help me move away from Oracle - or not?! (Oracle Community Tour EMEA - LVOUG...
Help me move away from Oracle - or not?!  (Oracle Community Tour EMEA - LVOUG...Help me move away from Oracle - or not?!  (Oracle Community Tour EMEA - LVOUG...
Help me move away from Oracle - or not?! (Oracle Community Tour EMEA - LVOUG...Lucas Jellema
 
Op je vingers tellen... tot 1000!
Op je vingers tellen... tot 1000!Op je vingers tellen... tot 1000!
Op je vingers tellen... tot 1000!Lucas Jellema
 
IoT - from prototype to enterprise platform (DigitalXchange 2022)
IoT - from prototype to enterprise platform (DigitalXchange 2022)IoT - from prototype to enterprise platform (DigitalXchange 2022)
IoT - from prototype to enterprise platform (DigitalXchange 2022)Lucas Jellema
 
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...Lucas Jellema
 
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...Lucas Jellema
 
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...Lucas Jellema
 
Introducing Dapr.io - the open source personal assistant to microservices and...
Introducing Dapr.io - the open source personal assistant to microservices and...Introducing Dapr.io - the open source personal assistant to microservices and...
Introducing Dapr.io - the open source personal assistant to microservices and...Lucas Jellema
 
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...How and Why you can and should Participate in Open Source Projects (AMIS, Sof...
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...Lucas Jellema
 
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...Lucas Jellema
 
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)Lucas Jellema
 
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...Lucas Jellema
 
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)Lucas Jellema
 
Tech Talks 101 - DevOps (jan 2022)
Tech Talks 101 - DevOps (jan 2022)Tech Talks 101 - DevOps (jan 2022)
Tech Talks 101 - DevOps (jan 2022)Lucas Jellema
 
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...Lucas Jellema
 
Cloud Native Application Development - build fast, low TCO, scalable & agile ...
Cloud Native Application Development - build fast, low TCO, scalable & agile ...Cloud Native Application Development - build fast, low TCO, scalable & agile ...
Cloud Native Application Development - build fast, low TCO, scalable & agile ...Lucas Jellema
 

More from Lucas Jellema (20)

Introduction to web application development with Vue (for absolute beginners)...
Introduction to web application development with Vue (for absolute beginners)...Introduction to web application development with Vue (for absolute beginners)...
Introduction to web application development with Vue (for absolute beginners)...
 
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...Making the Shift Left - Bringing Ops to Dev before bringing applications to p...
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...
 
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...Lightweight coding in powerful Cloud Development Environments (DigitalXchange...
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...
 
Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...
 
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...
 
Help me move away from Oracle - or not?! (Oracle Community Tour EMEA - LVOUG...
Help me move away from Oracle - or not?!  (Oracle Community Tour EMEA - LVOUG...Help me move away from Oracle - or not?!  (Oracle Community Tour EMEA - LVOUG...
Help me move away from Oracle - or not?! (Oracle Community Tour EMEA - LVOUG...
 
Op je vingers tellen... tot 1000!
Op je vingers tellen... tot 1000!Op je vingers tellen... tot 1000!
Op je vingers tellen... tot 1000!
 
IoT - from prototype to enterprise platform (DigitalXchange 2022)
IoT - from prototype to enterprise platform (DigitalXchange 2022)IoT - from prototype to enterprise platform (DigitalXchange 2022)
IoT - from prototype to enterprise platform (DigitalXchange 2022)
 
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...
 
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...
 
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...
 
Introducing Dapr.io - the open source personal assistant to microservices and...
Introducing Dapr.io - the open source personal assistant to microservices and...Introducing Dapr.io - the open source personal assistant to microservices and...
Introducing Dapr.io - the open source personal assistant to microservices and...
 
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...How and Why you can and should Participate in Open Source Projects (AMIS, Sof...
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...
 
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...
 
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
 
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...
 
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)
 
Tech Talks 101 - DevOps (jan 2022)
Tech Talks 101 - DevOps (jan 2022)Tech Talks 101 - DevOps (jan 2022)
Tech Talks 101 - DevOps (jan 2022)
 
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...
 
Cloud Native Application Development - build fast, low TCO, scalable & agile ...
Cloud Native Application Development - build fast, low TCO, scalable & agile ...Cloud Native Application Development - build fast, low TCO, scalable & agile ...
Cloud Native Application Development - build fast, low TCO, scalable & agile ...
 

Recently uploaded

The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfkalichargn70th171
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationkaushalgiri8080
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...aditisharan08
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningVitsRangannavar
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyFrank van der Linden
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...ICS
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsAlberto González Trastoy
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfkalichargn70th171
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEOrtus Solutions, Corp
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVshikhaohhpro
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxbodapatigopi8531
 

Recently uploaded (20)

The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learning
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The Ugly
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
 

Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14-03-2017)

  • 1. INTRODUCING NOSQL AND ONGODB Pom Bleeksma & Lucas Jellema 14 March 2017, Nieuwegein M
  • 2. AGENDA INTRODUCTION WHAT’S NOT SQL ABOUT NOSQL – HISTORY, BACKGROUND, USE CASES OVERVIEW OF NOSQL DATABASES INTRODUCING MONGODB GETTING STARTED WITH MONGODB – INSTALLATION, CONFIGURATION, ADMINISTRATION DEVELOPING WITH MONGODB – JAVA & NODE.JS HANDSON WORKSHOP
  • 3. TRADITIONAL APPROACH: ALL DATA IN ENTERPRISE RELATIONAL DATABASE
  • 4. TRADITIONAL APPROACH: WITHOUT MUCH REGARD FOR • What is it used for? • By whom, where, in what way, using which tools • Ratio between writes and reads • Ad hoc access? • What format is it in/should it be in? • What confidentiality & integrity is required? • How much of it? • How long relevant (hot vs cold vs dead)? • How fine grained and how accurate? • How much change?
  • 5. DATABASE: SINGLE POINT OF … SCALABILITY | AVAILABILITY | CONSISTENCY • Single Point of … • What goes where …
  • 6. RELATIONAL DATABASES • Based on relational model of data (E.F. Codd), a mathematical foundation • Uses SQL for query, DML and DDL • Transactions are ACID (Atomicity, Consistency, Isolation, Durability) • All or nothing • Constraint Compliant • Individual experience [in a multi-session environment] (aka concurrency) • Down does not hurt 6
  • 7. THE HOLY GRAIL OF NORMALIZATION • Normalize to prevent data redundancy, discrepancies (split brain) and storage waste • Pragmatic approaches - recognizing the fact that some data is read far more frequently than that it is created and modified • Materialized View • Virtual Column • Index • Sharding • Replication • Data Guard • Data Warehouse & Data Marts • Client Side Caching • In Memory Database
  • 8. ACID COMES AT A COST • Transaction results have to be persisted in order to guarantee D • Concurrency requires some degree of locking (and multi-versioning) in order to have I • Constraint compliance (unique key, foreign key) means all data hangs together (as do all transactions) in order to have C 8
  • 9. ACID COMES AT A COST • ACID means: • All data closely together (and sometimes far from applications and users) • All transactions centrally organized • All data persisted as part of the transaction • ACID means: • Hard to scale out (horizontally) • Oracle offers scale up with Real Application Clusters 9
  • 10. THE RELATIONAL MODEL IN PRACTICE • Traditional Relational Data Model has severe impact on physical disk performance • Transaction Log => Sequential Write (append to file) • Data Blocks require much more expensive Random Access disk writes • Indexes (B-Tree, Bitmap, …) are used to speed up query (read) performance • and slow down transactions • Relational data does not [always] map naturally to data format required in application (OO, JSON, XML) • Capability to join and construct ad-hoc queries across the entire data model is powerful • Declarative integrity constraints allow for strict enforcement of data quality rules • “the data may be non sensical, but at least it adheres to the rules, whatever the origin” • SQL has been embraced by huge numbers of tools & technologies and IT/Data professionals
  • 11. DATABASES RE-EVALUATED • Not all use cases require ACID (or can afford it) • Read only (product catalog for web shops) • Inserts only and no (inter-record) constraints • Big Data collected and “dumped” in Data Lake (Hadoop) for subsequent processing • High performance demands • Not all data needs structured formats or structured querying and JOINs • Entire documents are stored and retrieved based on a single key • Sometimes – scalable availability is more important than Consistency – and ACID is sacrificed • CAP-theorem states: Consistency [across nodes], Availability and Partition tolerance can not all three be satisfied 11
  • 12. USE CASE: WEBSHOP • Webshop – 1M visitors per day • Product catalog consists of millions of records • The web shop presents: product description, images, reviews, pricing details, related offerings, stock status • Products are added and updated and removed every day • Although most products do not change very frequently • Some vendors do bulk manipulation of product details Products Product updates Webshop visits - searches - product details - orders
  • 13. USE CASE: WEBSHOP TECHNOLOGY AND ARCHITECTURE CONSIDERATIONS Products Product updates Webshop visits - searches - product details - orders firewall Data manipulation Data Quality (enforcement) <10K transactions Batch jobs next to online Speed is nice Read only On line Speed is crucial XHTML & JSON > 1M visits
  • 14. Products Products USE CASE: WEBSHOP TECHNOLOGY AND ARCHITECTURE CONSIDERATIONS Products Webshop visits - searches - product details - orders firewall Data manipulation Data Quality (enforcement) <10K transactions Batch jobs next to online Speed is nice Read only On line Speed is crucial XHTML & JSON > 1M visits DMZ Read only JSON documents Images Text Search Scale Horizontally Products Nightly generation Product updates
  • 15. A FEW OBSERVATIONS • Redundancy • for performance, scalability, availability • Commodity hardware for scale out architecture • Data (documents) prepared for specific application usage (super MV) • Format • Search paths • Time lag in data modifications is accepted • Perhaps even desirable to offer a consistent view to webshop users • Transactions in one engine • ACID still in place • Transactions do not compete with reads • vying for the same physical resources
  • 16. NOSQL AND BASE • NoSQL arose because of performance and scalability challenges with Web Scale operations • NoSQL is a label for a wide variety of databases that lack some aspect of a true relational database • ACID-ness, SQL, relational model, constraints • The label has been used since 2009 • Perhaps NoREL would be more appropriate • Some well known NoSQL products are • Cassandra, MongoDB, Redis, CouchDB, … • BASE as alternative to ACID: • basically available, soft state, eventually consistent (after a short duration) 16
  • 17. TYPICAL FOR NOSQL • Focus on speed, availability and scalability • Horizontal scale out – distributed with load balancing and fail-over • No (predefined) Data Structure • Integrity primarily protected by application logic • Open Source (most offerings are, not all: MarkLogic) • Close(r) attention for how the data is used • Application oriented data format and search paths and specialized database per application (microservice, capability) • Similar to the switch from SOA to API/Microservice • Reads (far) more relevant than writes • Data redundancy & denormalization • No data access through SQL
  • 18. NOSQL MEANS: NO DATA ACCESS THROUGH SQL • However • Data Professionals and Developers speak SQL • Reporting, Dashboarding, ETL, BI tools speak SQL • There is no common query language across NoSQL products
  • 19. NO DATA ACCESS THROUGH SQL • However • Data Professionals and Developers speak SQL • Reporting, Dashboarding, ETL, BI tools speak SQL • There is no common query language across NoSQL products • Attempts from many vendors to create drivers that translate SQL statements into NoSQL commands for the specific target database • To protect existing investments in SQL – skills, tools, applications, reports, ..
  • 20. TYPES OF NOSQL DATABASES • Key-Value (from in-memory cache to hardware appliance – Memcached, Hazelcast, Coherence, Redis, Oracle NoSQL/Berkeley DB) • Wide Column Store (Cassandra, Google BigTable, HBase) • Document (JSON, XML, YAML and binary forms – MongoDB, CouchDB, Couchbase, MarkLogic) • Graph (networks of data, triplets, RDF stores – Neo4J) • Miscellaneous: object, tabular, tuple, elastic search index, Hadoop (hdfs) style, Kafka Topic
  • 21. (LEADING) NOSQL DATABASE PRODUCTS • MongoDB is (one of) the most popular (by any measure) • Cloud (only): • Google BigTable, • AWS Dynamo • Cache (in memory) • ZooKeeper, Redis, Memcached, … • Hadoop/HDFS • Oracle NoSQL (fka Berkeley DB)
  • 22.
  • 23. HISTORY OF MONGODB • 10gen – startup from 2007 in New York City • Developed database as component in Platform as a Service product • Initial release of MongoDB: 2009 • 10gen changed its name to MongoDB Inc. in 2013 • Offers enterprise support, MongoDB Atlas cloud service, MongoDB Compass, Connectors • Current stable release: 3.4.2 (3.5.2 is in preview) • Open Source - GNU Affero General Public License and the Apache License. • MongoDB == Humongous Database • Evolution of MongoDB is still pretty fast • Some crucial enterprise database capabilities were added fairly recently or are still lacking • MongoDB is number one NoSQL database in terms of popularity
  • 24. CORE ARCHITECTURE • MongoDB stores JSON documents in a binary format (BSON) • Documents are stored in collections (similar to row or records in tables) • Interaction is through JavaScript • taking the place of SQL for DML and query • Written in C++ • Has the V8 JavaScript engine included • Runs on Mac OS X, Windows, Solaris, and most flavors of Linux • Source code is on GitHub • Has GeoSpatial and Tekst indexes and search capabilities • MongoDB, Inc. officially supports drivers for C, C++, C#, Erlang, Java, Node.js, JavaScript, Perl, PHP, Python, Scala, and Ruby
  • 25. ARCHITECTUUR, INSTALLATIE, CONFIGURATIE, OPSLAG, SHELL • Mongo draait op windows en linux. • Windows install met msi file • linux install met tgz file. • Users en directories vrij te kiezen • Mongod --dbpath db1-- port 27001 • Default path=datadb, default port=27017 • Mongo [-- port hostname:27001] [--shell some_json_file] • Dbs, collections 25
  • 26. IMPORT EXPORT • Mongoimport • Json • Csv, tsv, maar hebben een descriptor file nodig • mongoexport -h -d -c -o -q • Leesbare output, minder geschikt voor backups
  • 27. CRUD • show dbs • use db • db.<collection>.find() • db.<collection>.find().pretty() • select: db.find({key:value,key:value}) • present: db.<collection>.find({key:value},{key:1,key:1,_id:0}) • explain: db.find({key:value,key:value}).explain("executionStats") • db.companies.insert({Company:"AMIS",Event:"MongoDB Workshop"}) • db.companies.update({Company:"AMIS"},{$set: {Location:"Nieuwegein"}}) • Geen commit! Ofwel, autocommit.
  • 28. CRUD • db.companies.remove({"Company" : "AMIS"}) • $unset -> Om een veld uit een document te verwijderen. • $inc -> om een numeriek veld te verhogen met een bepaalde waarde. Als het veld niet bestaat wordt het toegevoegd (met de waarde van de verhoging) • $push -> Om een waarde aan een array toe te voegen.
  • 29. INDEXING, EXPLAIN • _Id altijd geïndexeerd en unique • Indexen helpen bij zoeken • Indexen mogen compound zijn, dwz op meerdere keys gecombineerd, bv index op woonplaats+leeftijd • db.user.find().sort({ woonplaats:1, leeftijd: -1 }) De -1 betekent aflopend geïndexeerd. • Resultaten rechtstreeks uit index indien mogelijk (fat indexes in SQL) • db.user.find({woonplaats:”Zoetermeer”}).explain(“executionStats”) geeft een goed overzicht van het gevolgde plan
  • 30. REPLICA SETS • Replica sets zijn gedupliceerde collections • Beveilliging tegen uitval • De members besluiten onderling welk member primary is. Alleen daarop kan DML plaatsvinden • Altijd oneven aantal members, eventueel mbv een arbiter • Secondary members kunnen wel voor select gebruikt worden, als je eerst slaveOk() ingeeft • Mogen op dezelfde host draaien, mogen ook op verschillende hosts draaien. • writeConcern: Je kan expliciet aangeven op hoeveel members je DML uitgevoerd moet zijn voor je je prompt terug krijgt.
  • 31. SHARDING • Shard = splinter • Range based partitioning • Altijd index nodig op de shard key. Hoeft niet unique te zijn. • Iedere shard is een proces. Bij voorkeur ieder proces zijn eigen host. • Scale out proces • Metadata staat in aparte config db. Typically 3 stuks. • Clients verbinden met een mongos server. Dat is een instance zonder database die uit de config haalt waar de data zich bevindt. • Iedere shard kan bestaan uit replica sets.
  • 32. SHARDING • Data verdeelt zichzelf over chunks. • Chunks kunnen splitten wanneer ze te groot worden (typically 100MB maar is aan te passen naar wens) • Chunks kunnen migraten naar een andere chard. Mongo streeft ernaar elke shard evenveel chunks te laten hebben. • Tijdens migration kan DML gewoon doorgaan.
  • 33. BACKUP & RESTORE • Backup met mongodump • Creëert een dump directory met daaronder een directory per database en plaatst daarin 1 of meerdere .bson files, een binair formaat. • Kan vanaf ieder member van een replica set. Bij voorkeur eentje die het niet druk heeft. • Mongo kent geen transactions, dus backup is niet consistent. Het is maar afwachten wat erin staat als er veel DML plaatsvindt. • De tegenhanger heet mongorestore. • Restore van bestaande database lukt niet: duplicate key op _id
  • 34. MONGODB COMPASS (AKA NOSQL DEVELOPER) • The GUI for MongoDB • Visually explore data • Run ad hoc queries in seconds. • Interact with your data with full CRUD functionality • View and optimize your query performance. • Separately installed product • On Windows, OS X, Linux, … • Can run against local and remote MongoDB servers
  • 36. PROGRAMMING AGAINST MONGODB • MongoDB officially supports drivers for C, C++, C#, Erlang, Java, Node.js, JavaScript, Perl, PHP, Python, Scala, and Ruby • Note: there is also a Connector for Hadoop • Several REST APIs for MongoDB are available from the community • RESTHeart (Java), Eve (Python),, Crest (Node), AMID, Kule, DreamFactory Connect, create, update Connect, create, update
  • 37. MEAN STACK • Modelled after LAMP • End-to-End JavaScript/JSON • Term coined in 2013 (by MongoDB)
  • 38. INTERACTING WITH MONGODB FROM NODE.JS • MongoDB Node.js Driver 2.0 • Support for ECMAScript 6.0 – Promises for asynch • http://mongodb.github.io/node-mongodb-native/2.0/ • Using mongodb in a NodeJS application • npm install mongodb –save • var MongoClient = require('mongodb').MongoClient; Connect, create, update
  • 39. CONNECTING TO SERVER Connect, create, update var MongoClient = require('mongodb').MongoClient; var mongodbHost = '127.0.0.1'; var mongodbPort = '27017'; var mongodbDatabase = 'world'; var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase; MongoClient.connect(url, function(err, db) { console.log("Connected correctly to server."); // DO YOUR THING WITH MONGODB db.close(); console.log("Connection to database is closed."); }) //connect()
  • 40. RETRIEVE DATA (1) Connect, create, update var MongoClient = require('mongodb').MongoClient; var mongodbHost = '127.0.0.1'; var mongodbPort = '27017'; var mongodbDatabase = 'world'; var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase; MongoClient.connect(url, function(err, db) { console.log("Connected correctly to server."); db.collection('countries').find({},{"sort": [["area",-1]]}) .limit(20).toArray(function(err, results){ console.log("Name of Country Four " +results[3].name+ " and size: " +results[3].area); // using cursors to retrieve data sets in a controlled fashion // note: cursor implements NodeJS Stream - results can be piped var cursor = db.collection('countries').find({"continent":"Asia"}, {"sort": "name"}); cursor.count(function(err, count){ console.log("Country count in Asia: "+count); }); cursor.each(function(err, country){ console.log(country.name); })//each cursor
  • 41. RETRIEVE DATA (2) Connect, create, update var aggquery = [ {$sort: {area : -1}} , {$group:{ _id: '$continent' , largestCountry : {$first: "$name"} } } ]; var aggcursor = db.collection('countries').aggregate(aggquery); aggcursor.each(function(err, result){ if (err) { console.log(err); } else if (result) console.log(JSON.stringify(result)); }) //each aggcursor var ccursor = db.collection('countries').find({}); // the cursor returned from find and aggregate implements a NodeJS (readable) Stream ccursor.on('data', function(doc) { console.log(doc); }); ccursor.once('end', function() { console.log(“Out of countries. Time to move on"); });
  • 42. CONNECTING TO SERVER – ECMA 6 Connect, create, update var MongoClient = require('mongodb').MongoClient; var assert = require('assert'), co = require('co'); var mongodbHost = '127.0.0.1'; var mongodbPort = '27017'; var mongodbDatabase = 'world'; var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase; co(function*() { // Use connect method to connect to the Server var db = yield MongoClient.connect(url); console.log("Connected correctly to server"); // DO YOUR THING WITH MONGODB // Close the connection db.close(); }).catch(function(err) { console.log(err.stack); });
  • 43. RETRIEVE DOCUMENTS – ECMA 6 Connect, create, update co(function*() { // Use connect method to connect to the Server var db = yield MongoClient.connect(url); console.log("Connected correctly to server"); // find top 20 countries by size var results = yield db.collection('countries').find({} ,{"sort": [["area",-1]]}).limit(20).toArray(); console.log("Country One " +JSON.stringify(results[0])); console.log("Name of Country Four " +results[3].name + " and size: " +results[3].area); // use cursor to get the country count var cursor = db.collection('countries').find({"continent":"Asia"} , {"sort": "name"}); var count = yield cursor.count(); console.log("Country count in Asia: "+count); while (yield cursor.hasNext()){ var country = yield cursor.next(); console.log(country.name); }
  • 44. RETRIEVE DOCUMENTS (2) – ECMA 6 Connect, create, update // the largest country per continent var aggquery = [ {$sort: {area : -1}} , {$group:{ _id: '$continent' , largestCountry : {$first: "$name"} } } ]; var aggcursor = db.collection('countries').aggregate(aggquery); while (yield aggcursor.hasNext()){ var result = yield aggcursor.next(); console.log(JSON.stringify(result)); } // Close the connection db.close(); }).catch(function(err) { console.log(err.stack); });//co
  • 45. CREATE DOCS IN MONGODB – ES6 Connect, create, update // define two documents, any structure you like var doc = { "continent" : "Europe", "children" : [ {"name":"Belgium"}, {"name":"Luxemburg"}], "someVariable" : 19123, "andmore" : "2716NK" }; var doc2 = { "continent" : "Asia", "name" : "Johnny", "nrs" : [ {"name":"China"}, {"name":"India"}, {"name":"Buthan"}], "tree": {"branch": {"twig":{"leaf": 1}}} }; var nameOfCollection = "myDocs" var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]); console.log(">> "+result.insertedCount +" documents created into collection "+nameOfCollection); ...
  • 46. CREATE & UPDATE DOCS – ES6 Connect, create, update // define two documents, any structure you like var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“; var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]); console.log(">> "+result.insertedCount +" documents created); var cursor = db.collection(nameOfCollection).find(); while (yield cursor.hasNext()){ var doc = yield cursor.next(); console.log("Document: " +JSON.stringify(doc)); }// while cursor result = yield db.collection(nameOfCollection).updateOne( {"tree.branch.twig.leaf":1}, {$set: {name: "Hank", city:"Nieuwegein“ , "location.province":"Utrecht“ , "location.country":"The Netherlands“ , "tree.stem":5}}); console.log(">> Updated "+result.modifiedCount+" document(s)"); cursor = db.collection(nameOfCollection).find(); while (yield cursor.hasNext()){ var doc = yield cursor.next(); console.log("Document: " +JSON.stringify(doc)); }// while cursor
  • 47. CREATE, UPDATE AND DELETE – ES6 Connect, create, update // define two documents, any structure you like var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“; var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]); console.log(">> "+result.insertedCount +" documents created); var cursor = db.collection(nameOfCollection).find(); while (yield cursor.hasNext()){ var doc = yield cursor.next(); console.log("Document: " +JSON.stringify(doc)); }// while cursor result = yield db.collection(nameOfCollection).updateOne(...); ... result = yield db.collection(nameOfCollection).deleteMany({}); console.log(">> Deleted "+ result.deletedCount+" documents "); // execute command to drop the collection yield db.command({drop:nameOfCollection}); console.log(">> Dropped collection "+nameOfCollection); // Close the connection db.close(); console.log("Connection to database is closed.");
  • 48. INTERACTING WITH MONGODB FROM JAVA • MongoDB Java Driver 3.x • Supports Synchronous and Asynchronous operations • https://docs.mongodb.com/ecosystem/drivers/java/ • Using MongoDB Driver in a Java Application • Add dependency in Maven pom.xml file • Library JAR file mongo-java-driver-3.4.0.jar Connect, create, update
  • 49. CONNECTING TO SERVER package nl.amis.mongodb.countries; import java.net.UnknownHostException; import com.mongodb.MongoClient; import com.mongodb.client.MongoDatabase; public class AppConnect { public static void main(String[] args) throws UnknownHostException { String mongodbHost = "127.0.0.1"; Integer mongodbPort = 27017; String mongodbDatabase = "world"; MongoClient mongoClient = new MongoClient(mongodbHost, mongodbPort); try { System.out.println("Connected to server; now hook into database " + mongodbDatabase); MongoDatabase db = mongoClient.getDatabase(mongodbDatabase); System.out.println("Connected to database " + mongodbDatabase + " successfully"); } finally { mongoClient.close(); System.out.println("Closed mongodb connection"); } } } Connect, create, update
  • 50. RETRIEVE DATA ... import org.bson.Document; import com.mongodb.client.MongoCollection; public class AppConnect { public static void main(String[] args) throws UnknownHostException { ... Connect to world database // get all documents from the countries collection MongoCollection<Document> countries = db.getCollection("countries"); System.out.println("Got collection countries successfully"); System.out.println("Number of countries: " + countries.count()); System.out.println("All countries - name and continent"); for (Document doc : countries.find()) { // System.out.println(doc.toJson()); System.out.println(doc.get("name") + " (" + doc.get("continent") + ")"); }//for Connect, create, update
  • 51. RETRIEVE DATA (2) import com.mongodb.client.model.Sorts; import com.mongodb.client.model.Filters; import com.mongodb.Block; public class AppConnect { public static void main(String[] args) throws UnknownHostException { ... Connect to world database System.out.println( "Retrieve all countries in Asia order alphabetically by name and display full country document"); countries.find(Filters.eq("continent", "Asia")) .sort(Sorts.ascending("name")) .forEach(new Block<Document>() { @Override public void apply(final Document document) { System.out.println(document.toJson()); } }); Connect, create, update
  • 52. RETRIEVE DATA - AGGREGATION ... Block<Document> printBlock = new Block<Document>() { @Override public void apply(final Document document) { System.out.println("Country: " + document.toJson()); } }; ... System.out.println("List largest country in each continent"); countries.aggregate( Arrays.asList( Aggregates.sort(Sorts.descending("area")), Aggregates.group("$continent", Accumulators.first("Largest Country", "$name")) )) .forEach(printBlock); Connect, create, update
  • 53. RETRIEVE DATA – AGGREGATION (2) System.out.println("List name in uppercase,sorted by size, delta between birthdate and deathrate, " +" population density for countries in Oceania"); BsonArray subArgs=new BsonArray(); subArgs.add(new BsonString("$birthrate")); subArgs.add(new BsonString("$deathrate")); BsonArray divArgs=new BsonArray(); divArgs.add(new BsonString("$population")); divArgs.add(new BsonString("$area")); countries.aggregate (Arrays.asList( Aggregates.match(Filters.eq("continent", "Oceania")), Aggregates.sort(Sorts.descending("area")), Aggregates.project( Projections.fields( Projections.excludeId(), Projections.include("continent"), Projections.computed("name",Projections.computed("$toUpper", "$name")), Projections.computed("populationGrowthRate",Projections.computed("$subtract", subArgs)), Projections.computed("populationDensity", Projections.computed("$trunc", Projections.computed("$divide", divArgs))) )) )) .forEach(printBlock); Connect, create, update
  • 54. JAVA: CREATE DOCUMENTS IN MONGODB ... String nameOfCollection = "myDocs"; MongoCollection<Document> coll = db.getCollection(nameOfCollection); Document doc1 = new Document("continent", "Europe") .append("nrs", Arrays.asList(new Document("name", "Belgium"), new Document("name", "Luxemburg"))) .append("someVariable", 19123) .append("andmore", "kl;jdsfakhfsdahjfsdasjbsdahjbsdahgvsahjkZl;po"); Document doc2 = new Document("continent", "Asia").append("name", "Johnny") .append("nrs", Arrays.asList(new Document("name", "China"), new Document("name", "India"), new Document("name", "Buthan"))) .append("tree", new Document("branch", new Document("twig", new Document("leaf", 1)))); List<Document> documents = new ArrayList<Document>(); documents.add(doc1); documents.add(doc2); coll.insertMany(documents); System.out.println(">> Documents created into collection"); coll.find().forEach(printBlock); Connect, create, update
  • 55. JAVA: UPDATE AND DELETE DATA ... coll.updateOne( Filters.eq("tree.branch.twig.leaf", 1), Updates.combine( Updates.set("name", "Hank"), Updates.set("city", "Nieuwegein"), Updates.set("location.province", "Utrecht"), Updates.set("location.country", "The Netherlands"), Updates.set("tree.stemp", 5), Updates.currentDate("lastModified") ) ); System.out.println(">> Updated one document in collection " + nameOfCollection); coll.find().forEach(printBlock); DeleteResult dr = coll.deleteMany(Filters.exists("_id")); System.out.println(">> Deleted " + dr.getDeletedCount() + " documents from collection " + nameOfCollection); db.runCommand(new Document("drop", nameOfCollection)); System.out.println(">> Dropped collection " + nameOfCollection); Connect, create, update
  • 56. INTERACTING WITH MONGODB FROM JAVA – JSON  OO MAPPING • Libraries for mapping between BSON/JSON documents and Java Objects • Morphia - http://mongodb.github.io/morphia/1.3/getting-started/quick-tour/ • Jongo – Query in Java as in the MongoDB shell - http://jongo.org/ Connect, create, update
  • 57. HANDS ON WORKSHOP • Install MongoDB • Create Database, Create Collections and Documents • Retrieve Data • Import Database • More configuration and administration • Development: interacting with MongoDB from • Compass • Java • Node.js
  • 58. • Resources: https://github.com/lucasjellema/sig-nosql-mongodb • Blog: technology.amis.nl • Email: pom.bleeksma@amis.nl , lucas.jellema@amis.nl • : @lucasjellema • : lucas-jellema • : www.amis.nl, info@amis.nl +31 306016000 Edisonbaan 15, Nieuwegein