SlideShare a Scribd company logo
1 of 31
Download to read offline
mongoDB Project
Relational databases & Document-Oriented databases
Athens University of Economics and Business
Dpt. Of Management Science and Technology
Prof. Damianos Chatziantoniou
| lkoutsokera@gmail.com
| stratos.gounidellis@gmail.com
Lamprini Koutsokera
Stratos Gounidellis
BDSMasters
SQL Server vs. mongoDB
2
Description Microsoft’s relational DBMS One of the most popular
document stores
Database model Relational DBMS Document store
Implementation language C++ C ++
Data scheme yes schema-free
Triggers yes no
Replication methods yes, depending the SQL-Server Edition Master-slave replication
Partitioning methods tables can be distributed across Sharding
several files, sharding through
federation
References: [1]
From theory to practice
3
Tools
4
Required installations
5
References: [2]
1. Determine which MongoDB build you need.
2. Download MongoDB for Windows
3. Install MongoDB Community Edition.
4. Set up the MongoDB environment.
Mongodb installation
Required python packages installation
From software configuration to coding
6
Python coding [1] – table parsing
7
Part 1 - Queries and the Aggregation Pipeline [1]
. . .
load() function
mongo shell
prep.js file
References: [3]
Python coding [1] – table parsing
8
Query 1 : How many students in your database are currently taking at least 1 class (i.e. have a class with
a course_status of “In Progress”)?
Part 1 - Queries and the Aggregation Pipeline [2]
Query 2 : Produce a grouping of the documents that contains the name of each home city and the
number of students enrolled from that home city.
Python coding [1] – table parsing
9
Query 3 : Which hobby or hobbies are the most popular?
Part 1 - Queries and the Aggregation Pipeline [3]
the most popular
the top 5 popular
Python coding [1] – table parsing
10
Query 4 : What is the GPA (ignoring dropped
classes and in progress classes) of the best student?
Part 1 - Queries and the Aggregation Pipeline [4]
Query 5 : Which student has the largest number of
grade 10’s?
Python coding [1] – table parsing
11
Query 6 : Which class has the highest average
GPA?
Part 1 - Queries and the Aggregation Pipeline [5]
Query 7 : Which class has been dropped the most
number of times?
Python coding [1] – table parsing
12
Query 8 : Produce of a count of classes that have been COMPLETED by class type. The class type is found
by taking the first letter of the course code so that M102 has type M.
Part 1 - Queries and the Aggregation Pipeline [6]
Python coding [1] – table parsing
13
Query 9 : Produce a transformation of the documents so that the documents now have an additional boolean
field called “hobbyist” that is true when the student has more than 3 hobbies and false otherwise.
Part 1 - Queries and the Aggregation Pipeline [7]
Python coding [1] – table parsing
14
Query 10 : Produce a transformation of the documents so that the documents now have an additional field that
contains the number of classes that the student has completed.
Part 1 - Queries and the Aggregation Pipeline [8]
Python coding [1] – table parsing
15
Query 11 : Produce a transformation of the documents in
the collection so that they look like the following output.
The GPA is the average grade of all the completed
classes. The other two computed fields are the number of
classes currently in progress and the number of classes
dropped. No other fields should be in there. No other fields
should be present.
Part 1 - Queries and the Aggregation Pipeline [9]
Python coding [1] – table parsing
16
Query 12 : Produce a NEW collection (HINT: Use $out in the aggregation pipeline) so that the new documents
in this correspond to the classes on offer. The structure of the documents should be like the following output.
The _id field should be the course code.
The course_title is what it was before. The numberOfDropouts is the number of students who dropped out. The
numberOfTimesCompleted is the number of students that completed this class. The currentlyRegistered array is
an array of ObjectID’s corresponding to the students who are currently taking the class. Finally, for the students
that completed the class, the maxGrade, minGrade and avgGrade are the summary statistics for that class.
Part 1 - Queries and the Aggregation Pipeline [10]
Python coding [1] – table parsing
17
Part 1 - Queries and the Aggregation Pipeline [11]
Python coding [1] – table parsing
18
Part 2 - Python & MongoDB [1]
python_mongodb.py: Implement simple operations on mongo database.
Connect to mongo database and collection. Connect to mongo database and collection and insert
a record.
Python coding [1] – table parsing
19
Part 2 - Python & MongoDB [2]
Connect to mongo database and collection and insert
multiple records.
Connect to mongo database and collection and print
its content.
python_mongodb.py: Implement simple operations on mongo database.
Python coding [1] – table parsing
20
Part 2 - Python & MongoDB [3]
Connect to mongo database and collection and update
Its documents.
Connect to mongo database and collection and print
specific field.
python_mongodb.py: Implement simple operations on mongo database.
Python coding [1] – table parsing
21
Part 2 - Python & MongoDB [4]
Connect to mongo database and collection and convert the
collection to a dataframe.
Connect to mongo database and collection and import
data from a dataframe.
python_mongodb.py: Implement simple operations on mongo database.
Python coding [1] – table parsing
22
Part 2 - Python & MongoDB [5]
1. Clone this repository:
2. Install the required python packages.
3. Run python_mongodb.py to implement basic operations (insert_one, insert_many,
update, delete_one, delete_many, etc.) on mongodb.
python_mongodb.py: Implement simple operations on mongo database.
Python coding [1] – table parsing
23
Part 2 - Python & MongoDB [6]
Output
Python coding [1] – table parsing
24
Part 3 - MapReduce (Word Count) [1]
MapReduce 1 : Write a map reduce job on the
students collection similar to the classic word
count example. More specifically, implement a
word count using the course title field as the text.
In addition, exclude stop words from this list. You
should find/write your own list of stop words. (Stop
words are the common words in the English
language like “a”, “in”, “to”, “the”, etc.)
References: [4,5]
key value
[word1, 1]
[word2, 1]
[word3, 1]
[word2, 1]
[word4, 1]
. . . . . . . .
[wordx, 1]
Mapper
map
map
map
.
.
.
25
Part 3 - MapReduce (Word Count) [2]
course_title
[Predictive Modeling]
[MongoDB Operations]
[Hadoop and MapReduce]
[Data Mining]
[MongoDB Operations]
[Data Mining]
. . . . . . . . . . . . . . . . . . . . . .
Reducer
reduce
reduce
reduce
.
.
.
26
Part 3 - MapReduce (Word Count) [3]
key value
[word1, count1]
[word2, count2]
[word3, count3]
. . . . . . . . . . . . .
key value
[word1, {1, 1, 1, …} ]
[word2, {1, 1, 1, …} ]
[word3, {1, 1, 1, …} ]
. . . . . . . . . . . . . . . . .
Python coding [1] – table parsing
27
Part 3 - MapReduce (Average grade) [1]
MapReduce 2 : Write a map reduce job on the students
collection whose goal is to compute average
GPA scores for completed courses by
home city and by course type (M, B, P, etc.).
References: [4,5]
key value
[{home_city: Athina, course_type: M},
{count: 1, sum: 8}]
[{home_city: Chania, course_type: P},
{count: 1, sum: 6}]
[{home_city: Thyra, course_type: V},
{count: 1, sum: 3}]
[{home_city: Arta, course_type: M},
{count: 1, sum: 10}]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Mapper
map
map
map
.
.
.
28
Part 3 - MapReduce (Average grade) [2]
(course_code, home_city), grade
[(S201, Athina), 10]
[(M101, Mytilini), 9]
[(S202, Kavala), 3]
[(D102, Chania), 5]
[(P103, Athina), 6]
[(P101, Arta), 10]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Reducer
reduce
reduce
reduce
.
.
.
29
Part 3 - MapReduce (Average grade) [3]
key value
[{home_city: Athina, course_type: M},
[{count: 1, sum: 8}, [{count: 1, sum:
3}, [{count: 1, sum: 7}]
[{home_city: Chania, course_type: P},
{count: 1, sum: 6} , [{count: 1, sum:
5}, [{count: 1, sum: 4}]]
[{home_city: Thyra, course_type: V},
{count: 1, sum: 3} , [{count: 1, sum:
7}, [{count: 1, sum: 10}]]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
key value
[{home_city: Athina, course_type: M},
{count: 22, sum: 80}]
[{home_city: Chania, course_type: P},
{count: 8, sum: 100}]
[{home_city: Thyra, course_type: V},
{count: 47, sum: 300}]
[{home_city: Arta, course_type: M},
{count: 19, sum: 150}]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Finalizer
finalize
finalize
finalize
.
.
.
30
Part 3 - MapReduce (Average grade) [4]
key value
[{home_city: Athina, course_type: M},
{count: 22, sum: 80}]
[{home_city: Chania, course_type: P},
{count: 8, sum: 100}]
[{home_city: Thyra, course_type: V},
{count: 47, sum: 300}]
[{home_city: Arta, course_type: M},
{count: 19, sum: 150}]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
key value
[{home_city: Athina, course_type: M},
{avg: 8.5012}]
[{home_city: Chania, course_type: P},
{avg: 5.5314}]
[{home_city: Thyra, course_type: V},
{avg: 7.7713}]
[{home_city: Arta, course_type: M},
{avg: 6.4344}]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References
[1] Db-engines.com. (n.d.). System Properties Comparison Microsoft SQL Server vs. MongoDB vs. Oracle NoSQL
[online] Available at: https://db-engines.com/en/system/Microsoft+SQL+Server%3BMongoDB%3BOracle+NoSQL
[Accessed 5 May 2017].
[2] Install MongoDB Community Edition on Windows — MongoDB Manual 3.4.
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-windows/t [Accessed 2 May 2017].
[3] Aggregation Pipeline — MongoDB Manual 3.4 https://docs.mongodb.com/manual/core/aggregation-pipeline/
[Accessed 2 May 2017].
[4] Map-Reduce Examples — MongoDB Manual 3.4 https://docs.mongodb.com/manual/tutorial/map-reduce-
examples/ [Accessed 2 May 2017].
[5] Map-Reduce — MongoDB Manual 3.4 https://docs.mongodb.com/manual/core/map-reduce/ [Accessed 2 May
2017].
| lkoutsokera@gmail.com
| stratos.gounidellis@gmail.com
Lamprini Koutsokera
Stratos Gounidellis
BDSMasters

More Related Content

What's hot

CloudClustering: Toward a scalable machine learning toolkit for Windows Azure
CloudClustering: Toward a scalable machine learning toolkit for Windows AzureCloudClustering: Toward a scalable machine learning toolkit for Windows Azure
CloudClustering: Toward a scalable machine learning toolkit for Windows Azure
Ankur Dave
 
とある断片の超動的言語
とある断片の超動的言語とある断片の超動的言語
とある断片の超動的言語
Kiyotaka Oku
 
Deep dumpster diving 2010
Deep dumpster diving 2010Deep dumpster diving 2010
Deep dumpster diving 2010
RonnBlack
 

What's hot (20)

The Weather of the Century Part 3: Visualization
The Weather of the Century Part 3: VisualizationThe Weather of the Century Part 3: Visualization
The Weather of the Century Part 3: Visualization
 
Date and Time Module in Python | Edureka
Date and Time Module in Python | EdurekaDate and Time Module in Python | Edureka
Date and Time Module in Python | Edureka
 
CloudClustering: Toward a scalable machine learning toolkit for Windows Azure
CloudClustering: Toward a scalable machine learning toolkit for Windows AzureCloudClustering: Toward a scalable machine learning toolkit for Windows Azure
CloudClustering: Toward a scalable machine learning toolkit for Windows Azure
 
Vasia Kalavri – Training: Gelly School
Vasia Kalavri – Training: Gelly School Vasia Kalavri – Training: Gelly School
Vasia Kalavri – Training: Gelly School
 
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
 
Deep dive into deeplearn.js
Deep dive into deeplearn.jsDeep dive into deeplearn.js
Deep dive into deeplearn.js
 
Map reduce: beyond word count
Map reduce: beyond word countMap reduce: beyond word count
Map reduce: beyond word count
 
From Trill to Quill: Pushing the Envelope of Functionality and Scale
From Trill to Quill: Pushing the Envelope of Functionality and ScaleFrom Trill to Quill: Pushing the Envelope of Functionality and Scale
From Trill to Quill: Pushing the Envelope of Functionality and Scale
 
Weather of the Century: Design and Performance
Weather of the Century: Design and PerformanceWeather of the Century: Design and Performance
Weather of the Century: Design and Performance
 
GeoMesa on Apache Spark SQL with Anthony Fox
GeoMesa on Apache Spark SQL with Anthony FoxGeoMesa on Apache Spark SQL with Anthony Fox
GeoMesa on Apache Spark SQL with Anthony Fox
 
Enhancing Spark SQL Optimizer with Reliable Statistics
Enhancing Spark SQL Optimizer with Reliable StatisticsEnhancing Spark SQL Optimizer with Reliable Statistics
Enhancing Spark SQL Optimizer with Reliable Statistics
 
とある断片の超動的言語
とある断片の超動的言語とある断片の超動的言語
とある断片の超動的言語
 
Megadata With Python and Hadoop
Megadata With Python and HadoopMegadata With Python and Hadoop
Megadata With Python and Hadoop
 
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
 
Pyclustering tutorial - BANG
Pyclustering tutorial - BANGPyclustering tutorial - BANG
Pyclustering tutorial - BANG
 
Optimization
OptimizationOptimization
Optimization
 
GaianDB
GaianDBGaianDB
GaianDB
 
ThreeTen
ThreeTenThreeTen
ThreeTen
 
Deep dumpster diving 2010
Deep dumpster diving 2010Deep dumpster diving 2010
Deep dumpster diving 2010
 
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
 

Similar to MongoDB Project: Relational databases to Document-Oriented databases

Lecture 1 Pandas Basics.pptx machine learning
Lecture 1 Pandas Basics.pptx machine learningLecture 1 Pandas Basics.pptx machine learning
Lecture 1 Pandas Basics.pptx machine learning
my6305874
 

Similar to MongoDB Project: Relational databases to Document-Oriented databases (20)

mongoDB Project: Relational databases & Document-Oriented databases
mongoDB Project: Relational databases & Document-Oriented databasesmongoDB Project: Relational databases & Document-Oriented databases
mongoDB Project: Relational databases & Document-Oriented databases
 
PPT on Data Science Using Python
PPT on Data Science Using PythonPPT on Data Science Using Python
PPT on Data Science Using Python
 
Computer Science CS Project Matrix CBSE Class 12th XII .pdf
Computer Science CS Project Matrix CBSE Class 12th XII .pdfComputer Science CS Project Matrix CBSE Class 12th XII .pdf
Computer Science CS Project Matrix CBSE Class 12th XII .pdf
 
Tactical data engineering
Tactical data engineeringTactical data engineering
Tactical data engineering
 
Python Manuel-R2021.pdf
Python Manuel-R2021.pdfPython Manuel-R2021.pdf
Python Manuel-R2021.pdf
 
Lecture 1 Pandas Basics.pptx machine learning
Lecture 1 Pandas Basics.pptx machine learningLecture 1 Pandas Basics.pptx machine learning
Lecture 1 Pandas Basics.pptx machine learning
 
CBSE Class 12 Computer Science(083) Sample Question Paper 2020-21
CBSE Class 12 Computer Science(083) Sample Question Paper 2020-21CBSE Class 12 Computer Science(083) Sample Question Paper 2020-21
CBSE Class 12 Computer Science(083) Sample Question Paper 2020-21
 
SE-IT JAVA LAB SYLLABUS
SE-IT JAVA LAB SYLLABUSSE-IT JAVA LAB SYLLABUS
SE-IT JAVA LAB SYLLABUS
 
Python-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptxPython-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptx
 
Mmt 001
Mmt 001Mmt 001
Mmt 001
 
Chapter 16-spreadsheet1 questions and answer
Chapter 16-spreadsheet1  questions and answerChapter 16-spreadsheet1  questions and answer
Chapter 16-spreadsheet1 questions and answer
 
BDS_QA.pdf
BDS_QA.pdfBDS_QA.pdf
BDS_QA.pdf
 
More on Pandas.pptx
More on Pandas.pptxMore on Pandas.pptx
More on Pandas.pptx
 
XII - 2022-23 - IP - RAIPUR (CBSE FINAL EXAM).pdf
XII -  2022-23 - IP - RAIPUR (CBSE FINAL EXAM).pdfXII -  2022-23 - IP - RAIPUR (CBSE FINAL EXAM).pdf
XII - 2022-23 - IP - RAIPUR (CBSE FINAL EXAM).pdf
 
Python-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptxPython-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptx
 
Python for Data Analysis.pdf
Python for Data Analysis.pdfPython for Data Analysis.pdf
Python for Data Analysis.pdf
 
Python-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptxPython-for-Data-Analysis.pptx
Python-for-Data-Analysis.pptx
 
Python for data analysis
Python for data analysisPython for data analysis
Python for data analysis
 
CE344L-200365-Lab2.pdf
CE344L-200365-Lab2.pdfCE344L-200365-Lab2.pdf
CE344L-200365-Lab2.pdf
 
Object-Oriented Programming Using C++
Object-Oriented Programming Using C++Object-Oriented Programming Using C++
Object-Oriented Programming Using C++
 

Recently uploaded

一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
pyhepag
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理
cyebo
 
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotecAbortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
pyhepag
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
cyebo
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
pyhepag
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
RafigAliyev2
 

Recently uploaded (20)

一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
 
2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting
 
AI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdfAI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdf
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理
 
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotecAbortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
 
Slip-and-fall Injuries: Top Workers' Comp Claims
Slip-and-fall Injuries: Top Workers' Comp ClaimsSlip-and-fall Injuries: Top Workers' Comp Claims
Slip-and-fall Injuries: Top Workers' Comp Claims
 
2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call2024 Q1 Tableau User Group Leader Quarterly Call
2024 Q1 Tableau User Group Leader Quarterly Call
 
basics of data science with application areas.pdf
basics of data science with application areas.pdfbasics of data science with application areas.pdf
basics of data science with application areas.pdf
 
How I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prisonHow I opened a fake bank account and didn't go to prison
How I opened a fake bank account and didn't go to prison
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
 
Pre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxPre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptx
 
MALL CUSTOMER SEGMENTATION USING K-MEANS CLUSTERING.pptx
MALL CUSTOMER SEGMENTATION USING K-MEANS CLUSTERING.pptxMALL CUSTOMER SEGMENTATION USING K-MEANS CLUSTERING.pptx
MALL CUSTOMER SEGMENTATION USING K-MEANS CLUSTERING.pptx
 
Machine Learning for Accident Severity Prediction
Machine Learning for Accident Severity PredictionMachine Learning for Accident Severity Prediction
Machine Learning for Accident Severity Prediction
 
Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)
 
how can i exchange pi coins for others currency like Bitcoin
how can i exchange pi coins for others currency like Bitcoinhow can i exchange pi coins for others currency like Bitcoin
how can i exchange pi coins for others currency like Bitcoin
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
 
Easy and simple project file on mp online
Easy and simple project file on mp onlineEasy and simple project file on mp online
Easy and simple project file on mp online
 
Supply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflictSupply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflict
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
 

MongoDB Project: Relational databases to Document-Oriented databases

  • 1. mongoDB Project Relational databases & Document-Oriented databases Athens University of Economics and Business Dpt. Of Management Science and Technology Prof. Damianos Chatziantoniou | lkoutsokera@gmail.com | stratos.gounidellis@gmail.com Lamprini Koutsokera Stratos Gounidellis BDSMasters
  • 2. SQL Server vs. mongoDB 2 Description Microsoft’s relational DBMS One of the most popular document stores Database model Relational DBMS Document store Implementation language C++ C ++ Data scheme yes schema-free Triggers yes no Replication methods yes, depending the SQL-Server Edition Master-slave replication Partitioning methods tables can be distributed across Sharding several files, sharding through federation References: [1]
  • 3. From theory to practice 3
  • 5. Required installations 5 References: [2] 1. Determine which MongoDB build you need. 2. Download MongoDB for Windows 3. Install MongoDB Community Edition. 4. Set up the MongoDB environment. Mongodb installation Required python packages installation
  • 7. Python coding [1] – table parsing 7 Part 1 - Queries and the Aggregation Pipeline [1] . . . load() function mongo shell prep.js file References: [3]
  • 8. Python coding [1] – table parsing 8 Query 1 : How many students in your database are currently taking at least 1 class (i.e. have a class with a course_status of “In Progress”)? Part 1 - Queries and the Aggregation Pipeline [2] Query 2 : Produce a grouping of the documents that contains the name of each home city and the number of students enrolled from that home city.
  • 9. Python coding [1] – table parsing 9 Query 3 : Which hobby or hobbies are the most popular? Part 1 - Queries and the Aggregation Pipeline [3] the most popular the top 5 popular
  • 10. Python coding [1] – table parsing 10 Query 4 : What is the GPA (ignoring dropped classes and in progress classes) of the best student? Part 1 - Queries and the Aggregation Pipeline [4] Query 5 : Which student has the largest number of grade 10’s?
  • 11. Python coding [1] – table parsing 11 Query 6 : Which class has the highest average GPA? Part 1 - Queries and the Aggregation Pipeline [5] Query 7 : Which class has been dropped the most number of times?
  • 12. Python coding [1] – table parsing 12 Query 8 : Produce of a count of classes that have been COMPLETED by class type. The class type is found by taking the first letter of the course code so that M102 has type M. Part 1 - Queries and the Aggregation Pipeline [6]
  • 13. Python coding [1] – table parsing 13 Query 9 : Produce a transformation of the documents so that the documents now have an additional boolean field called “hobbyist” that is true when the student has more than 3 hobbies and false otherwise. Part 1 - Queries and the Aggregation Pipeline [7]
  • 14. Python coding [1] – table parsing 14 Query 10 : Produce a transformation of the documents so that the documents now have an additional field that contains the number of classes that the student has completed. Part 1 - Queries and the Aggregation Pipeline [8]
  • 15. Python coding [1] – table parsing 15 Query 11 : Produce a transformation of the documents in the collection so that they look like the following output. The GPA is the average grade of all the completed classes. The other two computed fields are the number of classes currently in progress and the number of classes dropped. No other fields should be in there. No other fields should be present. Part 1 - Queries and the Aggregation Pipeline [9]
  • 16. Python coding [1] – table parsing 16 Query 12 : Produce a NEW collection (HINT: Use $out in the aggregation pipeline) so that the new documents in this correspond to the classes on offer. The structure of the documents should be like the following output. The _id field should be the course code. The course_title is what it was before. The numberOfDropouts is the number of students who dropped out. The numberOfTimesCompleted is the number of students that completed this class. The currentlyRegistered array is an array of ObjectID’s corresponding to the students who are currently taking the class. Finally, for the students that completed the class, the maxGrade, minGrade and avgGrade are the summary statistics for that class. Part 1 - Queries and the Aggregation Pipeline [10]
  • 17. Python coding [1] – table parsing 17 Part 1 - Queries and the Aggregation Pipeline [11]
  • 18. Python coding [1] – table parsing 18 Part 2 - Python & MongoDB [1] python_mongodb.py: Implement simple operations on mongo database. Connect to mongo database and collection. Connect to mongo database and collection and insert a record.
  • 19. Python coding [1] – table parsing 19 Part 2 - Python & MongoDB [2] Connect to mongo database and collection and insert multiple records. Connect to mongo database and collection and print its content. python_mongodb.py: Implement simple operations on mongo database.
  • 20. Python coding [1] – table parsing 20 Part 2 - Python & MongoDB [3] Connect to mongo database and collection and update Its documents. Connect to mongo database and collection and print specific field. python_mongodb.py: Implement simple operations on mongo database.
  • 21. Python coding [1] – table parsing 21 Part 2 - Python & MongoDB [4] Connect to mongo database and collection and convert the collection to a dataframe. Connect to mongo database and collection and import data from a dataframe. python_mongodb.py: Implement simple operations on mongo database.
  • 22. Python coding [1] – table parsing 22 Part 2 - Python & MongoDB [5] 1. Clone this repository: 2. Install the required python packages. 3. Run python_mongodb.py to implement basic operations (insert_one, insert_many, update, delete_one, delete_many, etc.) on mongodb. python_mongodb.py: Implement simple operations on mongo database.
  • 23. Python coding [1] – table parsing 23 Part 2 - Python & MongoDB [6] Output
  • 24. Python coding [1] – table parsing 24 Part 3 - MapReduce (Word Count) [1] MapReduce 1 : Write a map reduce job on the students collection similar to the classic word count example. More specifically, implement a word count using the course title field as the text. In addition, exclude stop words from this list. You should find/write your own list of stop words. (Stop words are the common words in the English language like “a”, “in”, “to”, “the”, etc.) References: [4,5]
  • 25. key value [word1, 1] [word2, 1] [word3, 1] [word2, 1] [word4, 1] . . . . . . . . [wordx, 1] Mapper map map map . . . 25 Part 3 - MapReduce (Word Count) [2] course_title [Predictive Modeling] [MongoDB Operations] [Hadoop and MapReduce] [Data Mining] [MongoDB Operations] [Data Mining] . . . . . . . . . . . . . . . . . . . . . .
  • 26. Reducer reduce reduce reduce . . . 26 Part 3 - MapReduce (Word Count) [3] key value [word1, count1] [word2, count2] [word3, count3] . . . . . . . . . . . . . key value [word1, {1, 1, 1, …} ] [word2, {1, 1, 1, …} ] [word3, {1, 1, 1, …} ] . . . . . . . . . . . . . . . . .
  • 27. Python coding [1] – table parsing 27 Part 3 - MapReduce (Average grade) [1] MapReduce 2 : Write a map reduce job on the students collection whose goal is to compute average GPA scores for completed courses by home city and by course type (M, B, P, etc.). References: [4,5]
  • 28. key value [{home_city: Athina, course_type: M}, {count: 1, sum: 8}] [{home_city: Chania, course_type: P}, {count: 1, sum: 6}] [{home_city: Thyra, course_type: V}, {count: 1, sum: 3}] [{home_city: Arta, course_type: M}, {count: 1, sum: 10}] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mapper map map map . . . 28 Part 3 - MapReduce (Average grade) [2] (course_code, home_city), grade [(S201, Athina), 10] [(M101, Mytilini), 9] [(S202, Kavala), 3] [(D102, Chania), 5] [(P103, Athina), 6] [(P101, Arta), 10] . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 29. Reducer reduce reduce reduce . . . 29 Part 3 - MapReduce (Average grade) [3] key value [{home_city: Athina, course_type: M}, [{count: 1, sum: 8}, [{count: 1, sum: 3}, [{count: 1, sum: 7}] [{home_city: Chania, course_type: P}, {count: 1, sum: 6} , [{count: 1, sum: 5}, [{count: 1, sum: 4}]] [{home_city: Thyra, course_type: V}, {count: 1, sum: 3} , [{count: 1, sum: 7}, [{count: 1, sum: 10}]] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . key value [{home_city: Athina, course_type: M}, {count: 22, sum: 80}] [{home_city: Chania, course_type: P}, {count: 8, sum: 100}] [{home_city: Thyra, course_type: V}, {count: 47, sum: 300}] [{home_city: Arta, course_type: M}, {count: 19, sum: 150}] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 30. Finalizer finalize finalize finalize . . . 30 Part 3 - MapReduce (Average grade) [4] key value [{home_city: Athina, course_type: M}, {count: 22, sum: 80}] [{home_city: Chania, course_type: P}, {count: 8, sum: 100}] [{home_city: Thyra, course_type: V}, {count: 47, sum: 300}] [{home_city: Arta, course_type: M}, {count: 19, sum: 150}] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . key value [{home_city: Athina, course_type: M}, {avg: 8.5012}] [{home_city: Chania, course_type: P}, {avg: 5.5314}] [{home_city: Thyra, course_type: V}, {avg: 7.7713}] [{home_city: Arta, course_type: M}, {avg: 6.4344}] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  • 31. References [1] Db-engines.com. (n.d.). System Properties Comparison Microsoft SQL Server vs. MongoDB vs. Oracle NoSQL [online] Available at: https://db-engines.com/en/system/Microsoft+SQL+Server%3BMongoDB%3BOracle+NoSQL [Accessed 5 May 2017]. [2] Install MongoDB Community Edition on Windows — MongoDB Manual 3.4. https://docs.mongodb.com/manual/tutorial/install-mongodb-on-windows/t [Accessed 2 May 2017]. [3] Aggregation Pipeline — MongoDB Manual 3.4 https://docs.mongodb.com/manual/core/aggregation-pipeline/ [Accessed 2 May 2017]. [4] Map-Reduce Examples — MongoDB Manual 3.4 https://docs.mongodb.com/manual/tutorial/map-reduce- examples/ [Accessed 2 May 2017]. [5] Map-Reduce — MongoDB Manual 3.4 https://docs.mongodb.com/manual/core/map-reduce/ [Accessed 2 May 2017]. | lkoutsokera@gmail.com | stratos.gounidellis@gmail.com Lamprini Koutsokera Stratos Gounidellis BDSMasters