JSON Data Modeling in Document Database

Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2021. All rights reserved.
April 2022
Matthew D. Groves | Product Marketing Manager
JSON Data Modeling

Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2021. All rights reserved. 2
Modeling Data in a Relational World
RDBMS table-based modeling
Billing
Connections
Purchases
Contacts
Customer

01/
02/
03/
04/
05/
Why NoSQL?
JSON Data Modeling
Accessing Data
Migrating Data
Summary / Q&A
Agenda

1 Why NoSQL?

NoSQL Landscape
• Couchbase
• DynamoDB
• CosmosDB
• Redis
• Aerospike
• Couchbase
• MongoDB
• DynamoDB
• CosmosDB
Key-Value Document
• Cassandra
• Hbase
• CosmosDB
• Datastax
• Neo4J
• OrientDB
• CosmosDB
Wide Column Graph
🔑 📄 🔢 🗺
Broad categories, many are "multi-model"

NoSQL Landscape
• Couchbase
• MongoDB
• DynamoDB
• CosmosDB
Document
📄
Broad categories, many are "multi-model"
• Store data as JSON (or XML, etc)
• Get by key
• Set by key
• Replace by key
• Delete by key
• Query/index

What's NoSQL?

Why NoSQL? Scalability

Why NoSQL? Flexibility
{
"name" : "matt groves"
}
{
"firstName" : "jeff",
"lastName" : "morris"
}
DocumentKey: 120902
DocumentKey: 930912

Why NoSQL? Availability

Why NoSQL? Fast

Use Cases for NoSQL
• Caching
• Session
• User profile
• Catalog
• Content management
• Personalization
• Customer 360
• IoT
• Inventory
• Communication
• Gaming
• Advertising
• Travel booking
• Loyalty programs
• Fraud monitoring
• Social media
• Finance
• Health Care

Use Cases

2 JSON Data Modeling

Properties of Real-World Data

Modeling Data in a Relational World
RDBMS table-based modeling
Billing
Connections
Purchases
Contacts
Customer

CustomerID Name DOB
CBL2015 Jane
Smith
1990-01-30
Table: Customer
{
"Name" : "Jane Smith",
"DOB" : "1990-01-30”
}
Customer Document
Key: CBL2015

©2017 Couchbase Inc. 18
CustomerID Name DOB
CBL2015 Jane
Smith
1990-01-30
Table: Customer
{
"DOB" : "1990-01-30",
"Purchases" : [
{
"item" : "laptop",
"amount" : 1499.99,
"date" : "2019-03",
}
]
}
Customer Document
Key: CBL2015
CustomerID Item Amount Date
CBL2015 laptop 1499.99 2019-03
Table: Purchases

CustomerID Name DOB
CBL2015 Jane
Smith
1990-01-30
Table: Customer
{
"DOB" : "1990-01-30",
"Purchases" : [
{
"item" : "laptop",
"amount" : 1499.99,
"date" : "2019-03",
},
{
"item" : "phone",
"amount" : 99.99,
"date" : "2018-12"
}
]
}
Customer Document
Key: CBL2015
CustomerID Item Amount Date
CBL2015 laptop 1499.99 2019-03
CBL2015 phone 99.99 2018-12
Table: Purchases

CustomerID ConnId Relation
CBL2015 XYZ987 Brother
CBL2015 SKR007 Father
Table: Connections
{
"DOB" : "1990-01-30",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5827-2842-...",
"expiry" : "2019-03"
}, ...
],
"Connections" : [
{
"ConnId" : "XYZ987",
"Relation" : "Brother"
},
{
"ConnId" : "SKR007",
"Relation" : "Father"
}
}
Customer Document
Key: CBL2015

©2017 Couchbase Inc. 21
{
"DOB" : "1990-01-30",
"cardnum" : "5827-2842…",
"expiry" : "2019-03",
"cardType" : "visa",
"Connections" : [
{
"CustId" : "XYZ987",
},
{
"CustId" : "SKR007",
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 }
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
Customer Document
Key: CBL2015
CustomerID Name DOB Cardnum Expiry CardType
CBL2015 Jane
Smith
1990-01-30 5827-2842… 2019-03 visa
CustomerID item amt
CBL2015 mac 2823.52
CBL2015 ipad2 623.52
CustomerID ConnId Name
CBL2015 XYZ987 Joe
Smith
CBL2015 SKR007 Sam
Smith
Contacts
Customer
Connections
Purchases

{
"Name" : "Bob Jones",
"DOB" : "1980-01-29",
"Billing" : [
{
"type" : "visa",
"cardnum" : "5927-2842-2847-3909",
"expiry" : "2020-03"
},
{
"type" : "master",
"cardnum" : "6273-2842-2847-3909",
"expiry" : "2019-11"
}
],
"Connections" : [
{
"CustId" : "XYZ987",
},
{
"CustId" : "PQR823",
}
],
"Purchases" : [
{ "id":12, item: "mac", "amt": 2823.52 },
{ "id":19, item: "ipad2", "amt": 623.52 }
]
}
Customer Document
Key: CBL2016
CustomerID Name DOB
CBL2016 Bob Jones 1980-01-29
CustomerID Type Cardnum Expiry
CBL2016 visa 5927… 2020-03
CBL2016 master 6273… 2019-11
CustomerID item amt
CBL2016 mac 2823.52
CBL2016 ipad2 623.52
CustomerID ConnI
d
Name
CBL2016 XYZ987 Joe
Smith
CBL2016 SKR007 Sam
Smith
Contacts
Customer
Billing
Connections
Purchases

Versioning approach 1: Version Numbers
{
"version" : 1
}
{
"firstName" : "jeff",
"lastName" : "morris",
"version" : 2
}
Document Key:
120902
Document Key:
930912

Versioning approach 2: Big Bang Re-versioning
{
}
{
"firstName" : "matt",
"lastName" : "groves
}
Document Key:
120902
Document Key:
120902

Versioning approach 3: Cooperative Re-versioning
{
}
{
"firstName" : "matt",
"lastName" : "groves",
}
DocumentKey: 120902
DocumentKey: 120902
Web application

Modeling tools
• Hackolade
• Erwin DM NoSQL
• https://jsoneditoronline.org

3 Accessing Data

Key/Value
public async Task<ShoppingCart> GetCartById(string id)
{
var cart = await _collection.GetAsync(id);
return cart.ContentAs<ShoppingCart>();
}
public async Task CreateShoppingCart()
{
await _collection.InsertAsync(
Guid.NewGuid().ToString(),
new ShoppingCart { . . . }
);
}

Key/Value: Recommendations for keys
• Natural Keys
• Account number, driver license number, ISBN
• Human Readable
• "user-123" vs "0be12ae9-1b1d-4724-9706-2977cc134daf"
• Deterministic
• "matt" -> "matt::blogs" -> "blog::csharp_9_features"
• Semantic

Key/Value: Example keys
1. User logs in with username and password
• Lookup "author::<username>" in the database
2. Once logged in, save username to session state.
3. User wants to view their blogs.
• Lookup "author::<username>::blogs" in the database
4. Document contains list of "slugs"
5. User wants to view a specific blog.
• Lookup "blog::<slug>" in the database
6. User wants to view all comments for this post
• Lookup "blog::<slug>::comments" in the database

Modeling your data: Strategies / rules of thumb
• Relationship is one-to-one or one-
to-many
• Store related data as nested
objects
{
"DOB" : "1990-01-30",
"Purchases" : [
{
"item" : "laptop",
"amount" : 1499.99,
"date" : "2019-03",
},
{
"item" : "phone",
"amount" : 99.99,
"date" : "2018-12"
}
]
}

• Relationship is many-to-one or
many-to-many
• Store related data as separate
documents
{
"DOB" : "1990-01-30",
"Connections" : [
"XYZ987",
"PQR823",
"PQR828"
]
}

• Data reads are mostly parent fields
• Store children as separate
documents
{
"DOB" : "1990-01-30",
"Connections" : [
"XYZ987",
"PQR823",
"PQR828"
]
}

• Data reads are mostly parent +
child fields
• Store children as nested objects
{
"DOB" : "1990-01-30",
"Purchases" : [
{
"item" : "laptop",
"amount" : 1499.99,
"date" : "2019-03",
},
{
"item" : "phone",
"amount" : 99.99,
"date" : "2018-12"
}
]
}

• Data writes are mostly parent or
child (not both)
• Store children as separate
documents
{
"DOB" : "1990-01-30",
"Connections" : [
"XYZ987",
"PQR823",
"PQR828"
]
}

• Data writes are mostly parent and
child (both)
• Store children as nested objects
{
"DOB" : "1990-01-30",
"Purchases" : [
{
"item" : "laptop",
"amount" : 1499.99,
"date" : "2019-03",
},
{
"item" : "phone",
"amount" : 99.99,
"date" : "2018-12"
}
]
}

If … Then …
Relationship is one-to-one or one-to-many Store related data as nested objects
Relationship is many-to-one or many-to-many Store related data as separate documents
Data reads are mostly parent fields Store children as separate documents
Data reads are mostly parent + child fields Store children as nested objects
Data writes are mostly parent or child (not
both)
Store children as separate documents
Data writes are mostly parent and child (both) Store children as nested objects

Subdocument access
{
"username": "mgroves",
"profile": {
"phoneNumber": "123-456-7890",
"address": {
"street": "123 main st",
"city": "Grove City",
"state": "Ohio"
}
}
}

Other ways to access data (Couchbase)
Key-Value
(CRUD)
SQL++
(SQL Query)
Full Text
(Search)
Documents
Indexes Indexes
Mobile
(iOS/
Android)
Analytics
(Query)
Sync
Gateway
SQL++

SQL++

Understanding your Query Plan
"recommended_indexes": {
"indexes": [{
"index_statement":
"CREATE INDEX adv_name ON `default`:`travel-sample`.`inventory`.`airline`(`name`)",
"keyspace_alias": "airline_a",
"recommending_rule": "Index keys follow order of predicate types: 8. like."
}]
}

Full Text Search

Accessing your data
Strategies and recommendations
Concept Strategies & Recommendations
Key-Value Operations provide the best
possible performance
• Create an effective key naming strategy
• Create an optimized data model
Full Text Search is well-suited to text • Facets / ranges / geography
• Language aware
• Inverted index
N1QL queries provide the most flexibility –
everything else
• Query data regardless of how it is modeled
• Good indexing is vital
• B-Tree

4 Migrating Data

How do you migrate?
A spectrum of approaches
1. Rewrite: No migration, write the whole thing over
2. Redesign Schema: Keep your business logic, rewrite your data
layer and schema, totally redesign your schema with a NoSQL-
optimized model
3. Refactor First: Keep everything but refactor your data logic
and RDBMS schema into a NoSQL-optimized model
4. Optimize Later: Host your schema with as few changes as
possible, get the application running on the new technology,
refactor/optimize the schema as necessary for performance
5. Just Host It: Host your schema with as few changes as
possible.
Risk
Effort

Migration options
Tools

Migration options
Build Your Own (BYO)

Migration options: KISS (level 5)
"Level 5" = "Just Host it"
Export
Transform
Import
NoSQL (optimized)
Relational
NoSQL
(raw)

Migration options
KISS (levels 4,3,2)
Export
Transform
Import
NoSQL
Relational
NoSQL (optimized)
"staging"

Migration Recommendations
Alignment

SqlServerToCouchbase Demo

5 Summary

Pick the right
application

Proof of Concept

Match the data
access method to
requirements

Next Steps
• Couchbase playground:
• https://couchbase.live
• Couchbase Capella free trial:
• https://cloud.couchbase.com/sign-up
• SqlServerToCouchbase:
• https://github.com/mgroves/SqlServerToCouchbase

JSON Data Modeling in Document Database

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to JSON Data Modeling in Document Database

Similar to JSON Data Modeling in Document Database (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

JSON Data Modeling in Document Database