Data analytics can offer insights into your business and help take it to the next level. In this talk you'll learn about MongoDB tools for building visualizations, dashboards and interacting with your data. We'll start with exploratory data analysis using MongoDB Compass. Then, in a matter of minutes, we'll take you from 0 to 1 - connecting to your Atlas cluster via BI Connector and running analytical queries against it in Microsoft Excel. We'll also showcase the new MongoDB Charts product and you'll see how quick, easy and intuitive analytics can be on the MongoDB platform without flattening the data or spending time and effort on complicated and fragile ETL.
4. Data Analytics with MongoDB
Custom Code +
Charting Libraries
ETL +
3rd Party BI
Tools
MongoDB BI Connector +
3rd Party BI Tools
MongoDB
Charts
MongoDB
Compass
5. But First, Setup
MongoDB Atlas Cluster
MongoDB Connector for BI
Charts Server
MongoDB Compass
https://github.com/breinero/workshop
7. MongoDB BI Connector
Visualize and explore MongoDB
data in SQL-based BI tools:
Automatically discovers the schema
Translates complex SQL statements
issued by the BI tool into MongoDB
aggregation queries
Converts the results into a tabular format
for rendering inside the BI tool
11. Demo
● use $ mongo to examine data in MongoDB
● use $ mysql to examine tabular representation of collections
● explain()
● mysql -h dataanalyticsworkshop-biconnector-sshrq.mongodb.net -P
27015 -u demo -p --default-auth=mongosql_auth
12. Exercise 1: Analysing AirBnB Data
with BI Connector
Make a DSN connection to MongoDB
Import Data Into Excel
Identify the most expensive neighborhood group in Austin
Determine average nightly price for an Airbed, per neighborhood
13. MongoDB BI Connector - When to Use
Want to speak SQL to MongoDB
Multi data sources (not just MongoDB)
Business analysts
Reporting only
Powerful but you lose some benefits of schema flexibility
15. Options for Visualizing MongoDB Data
Custom Code + Charting
Libraries
ETL + 3rd Party BI ToolsBI Connector + 3rd Party
BI Tools
16. Wouldn’t it be nice if...
You could visualize your MongoDB Data…
without needing to write your own code
without needing to move your data into a different repository
without needing to purchase and configure third-party tools
without losing the richness of the Document Model
17. MongoDB Charts beta
The fastest way to build
visualizations over your
MongoDB data
Built for the MongoDB Document
Model
Visualize live data from on-prem
or Atlas DB
18. Charts Basic Concepts
A data source is a reference to a MongoDB collection or view that
contains data you want to visualize.
A chart is a visualization of data from a single data source.
A dashboard is a collection of charts which you manage as a unit
(name, layout, sharing)
19. What to Expect
Charting Capabilities
Common chart types
Aggregation functions
Filtering
Sample Mode
Binning
Sorting
Document Model Support
Type handling
Polymorphic collections
Nested documents
Array reductions
20. Sharing and Permissions
All users log onto Charts with their own account
• Users can be managed by anyone with the UserAdmin role
Any user can add a data source
• They must have a valid connection URI to connect to the MongoDB instance
• A data source owner can choose to share with nobody, specific people or
everybody
Any user can add a dashboard
• By default, the dashboard is only visible to the creator
• A dashboard owner can choose to share with nobody, specific people or
everybody
21. Exercise 2: Analysing AirBnB Data
with Charts
Add a Data Source
Create a Dashboard
Add some charts
Multi-Series Stacked Bar Chart
Coloured Bar Chart
Review Scores Histogram
Area Chart with Binning
22. The fastest way to build visualizations over your MongoDB
data
Ad hoc analyses
Benefit from the Document Model
Collaboration
Self-service
Intuitive enough for domain experts, non-devs to use!
25. Rich Queries
Point | Range | Geospatial | Faceted Search | Aggregations | JOINs | Graph Traversals
JSON Documents Tabular Key-Value Text GraphGeospatial
MongoDB Aggregation Framework
26. MongoDB
{ customer_id : 1,
first_name : "Mark",
last_name : "Smith",
city : "San Francisco",
phones: [ {
number : "1-212-777-1212",
type : "work"
},
{
number : "1-212-777-1213",
type : "cell"
}]
……...
Expressive
Queries
Find anyone with phone # “1-212…”
Check if the person with number “555…” is on the “do not call” list
Geospatial
Find the best offer for the customer at geo coordinates of 42nd St.
and 6th Ave
Text Search Find all tweets that mention the firm within the last 2 days
Aggregation
Count and sort number of customers by city, compute min, max, and
average spend
Native Binary
JSON Support
Add an additional phone number to Mark Smith’s record without
rewriting the document
Update just 2 phone numbers out of 10
Sort on the modified date
JOIN
($lookup)
Query for all San Francisco residences, lookup their transactions,
and sum the amount by person
Graph Queries
($graphLookup)
Query for all people within 3 degrees of separation from Mark
Rich query functionality
29. Developer / Data Analyst Tool
Data management and manipulation
document view
table view
Visual schema analyzer
with query builder
export to language
Aggregation pipeline builder
A good place to start
30. Exercise 3: Building Powerful
Aggregations
Explore data by doing some basic finds
Learn how to do aggregations you usually do in SQL
$match
$group
Learn how to do aggregations you cannot do in SQL
$objectToArray
$graphlookup
$geonear
31. Find or Count Specific Documents
4. Find how many listings have a real bed and are greater than
$100 but less than $200 per night by filling in the blanks
db.___.find( {
price: { ___: ___, $lt: ___ } } ,
{ ___ : "Real Bed" }).count()
db.austinListingsAndReviews.find( {
"price": { $gt: 100, $lt: 200 } } ,
{"bed_type" : "Real Bed"}).count()
Answer: 2892
32. Minimum price per night by number
of beds
Take a look at this example aggregation which computes minimum
price per night by number of beds
db.austinListingsAndReviews.aggregate( [
{
$group: {
_id: "$beds",
avg_by_beds: { $min: "$price" }
}
},
{ $sort: { avg_by_beds: 1 } }
] )
33. Average price per night by room type
Construct a very similar query to find average price per night
by room type by filling in the blanks.
db.austinListingsAndReviews.aggregate( [
{
$group: {
_id: "___",
avg_by_roomtype: { $avg: "___" }
}
}}
] )
34. ANSWER: Average price per night by
room type
Construct a very similar query to find average price per night
by room type by filling in the blanks.
db.austinListingsAndReviews.aggregate( [
{
$group: {
_id: "$room_type",
avg_by_roomtype: { $avg: "$price" }
}
}}
] )
35. Average price per night by suburb
Let’s practice some more.
This time write the query from scratch to find the average
price per night by suburb. Examine the document structure
first.
Hint: suburb is a field within a subdocument address; use dot
notation to reference it.
36. ANSWER: Average price per night by
suburb
db.austinListingsAndReviews.aggregate( [
{
$group: {
_id: "$address.suburb",
avg_by_suburb: { $avg: "$price" }
}
}
] )
37. Count and average price per night
Compute the count of properties and average price per night by
suburbs and number of bedrooms by filling in the blanks.
db.austinListingsAndReviews.aggregate([
{"$group" :
{
_id:{suburb:"$___", bedrooms:"$___"},
___:{$sum:1},
avg:{$___:"$price"}}
}
])
38. ANSWER: Count and average price
per night
db.austinListingsAndReviews.aggregate([
{"$group" :
{
_id:{suburb:"$address.suburb", bedrooms:"$bedrooms"},
count:{$sum:1},
avg:{$avg:"$price"}}
}
])
39. Add $match Stage
Use the query from previous step and add a $match stage to only
show those properties that have a pool
db.austinListingsAndReviews.aggregate([
{ $match : {amenities: "Pool"} },
{"$group" :
{
_id:{suburb:"$address.suburb",bedrooms:"$bedrooms"},
count:{$sum:1},
avg:{$avg:"$price"}}
}
])
40. Learn how to do aggregations you
cannot do in SQL
We want to determine average review scores for all properties
in Austin based on some amenities.
Remember that review_scores is a sub document. Luckily we have
$objectToArray to transform our object into an array which we
can then $unwind and $group.
41. Average review scores for all
properties in Austin with amenities
db.austinListingsAndReviews.aggregate([
{ $match : {
amenities: { $in: ["Cable TV", "Garden or backyard",
"Coffee maker"] } } },
{ $project: { scores: { $objectToArray: "$review_scores" } } },
{ $unwind: "$scores" },
{ $group: { _id: "$scores.k", average_score: { $avg:
"$scores.v" } } }
])
42. Compare two neighbourhoods
Compare two neighbourhoods - South Lamar and West Campus - and
decide based on average review where you would rather stay.
Assume you also want a place that has pool, wifi, and
air-conditioning, under $200
Hint: You will need to look up how to use $and operator for this
45. $geoNear
Use $geoNear to find airbnbs near location of your choice (or
near Alamo Drafthouse - 30.2988474,-97.7064386) and add the
same search criteria we used in in the prior exercise to only
show airbnbs that have pool, wifi, and air-conditioning, under
$200
47. $graphLookup
Let’s say we want to build a recommendation engine based on
users that reviewed the same properties.
We can use $graphLookup to run a recursive query that says:
“for a given reviewer, e.g. 7538 let’s find all listings
reviewed by users who also reviewed the same listings as 7538”
49. Now try $graphlookup yourself
Some host in Austin have multiple listings, for example,
host_id:100208662. Construct a $graphLookup query that finds
all the listings by that host.
51. Exploratory data analysis
Data preparation & basic manipulation
Data ingestion via JSON or CSV import
Day-to-day development/operations
Adding and understanding indexes
Adding validation rules
Authoring & troubleshooting aggregation pipelines
Viewing real-time server stats
10,000 → 1ft view of data
53. Business Intelligence, Analytics, Machine Learning
Process data in MongoDB with the massive parallelism
of Spark, it's machine learning libraries, and streaming
API
● Process data “in place”, avoiding the
latency otherwise required by an
incremental ETL task.
● Reduced Operational Complexity and Faster
Time-To-Analytics
● Aggregation pre-filtering in combination with
secondary indexing means that an analytics
query only draws that data required
● Multiple Language APIs
54. JSON
JSON
JSON
JSON
JSON
JSON
JSON
JSON
JSON
JSON
JSON
Business Intelligence, Analytics, Machine Learning
Process data in MongoDB with the massive
parallelism of Spark, it's machine learning libraries,
and streaming API
● Process data “in place”, avoiding the latency
otherwise required by an incremental ETL task
● Aggregation pre-filtering in combination
with secondary indexing means that an
analytics query only draws that data
required
● Reads from secondaries isolate analytics
workload from business critical operations
● Shard aware for data locality
55. WRITE
READ
Primary
2ndary
2ndary
Business Intelligence, Analytics, Machine Learning
Process data in MongoDB with the massive
parallelism of Spark, it's machine learning libraries,
and streaming API
● Process data “in place”, avoiding the latency
otherwise required by an incremental ETL task
● Aggregation pre-filtering in combination with
secondary indexing means that an analytics
query only draws that data required
● Reads from secondaries isolate analytics
workload from business critical operations
● Shard aware for data locality
56. Business Intelligence, Analytics, Machine Learning
Process data in MongoDB with the massive
parallelism of Spark, it's machine learning libraries,
and streaming API
● Process data “in place”, avoiding the latency
otherwise required by an incremental ETL task
● Aggregation pre-filtering in combination with
secondary indexing means that an analytics
query only draws that data required
● Reads from secondaries isolate analytics
workload from business critical operations
● Multiple language APIs