Introduction to data modeling on NoSQL by Ido Friedman from Perion.

1. NoSQL Data Modeling

2. IdoFriedman.yml
Name: Ido Friedman,
Past:”SQL Server consultant,Instructor,Team Leader”
Present:”Data engineer and Architect,
Elasticsearch,CouchBase,MongoDB,Python”,…]
WorkPlace:Perion
WhenNotWorking:@Sea

3. Let’s talk
• What is the role of data modeling
• What does data modeling effect

4. Data models
Document Columnar Graph
Relational New SQL*
And
more..
Document

5. Data Domains
On line
Batch
Real time
analytics
Micro
batch
Streaming

6. Schema and structure
Schema
free
Structured
Unstruc
tured
Semi
Structured
• Who needs schemas?
• Schema description

7. Normalization/De-Normalization
• Born to reserve storage and keep data integrity (RI)
• Resolve data joining issues
• Performance aspects
Is it still relevant???

8. Normalization example
{"_index": "user_profiles",
"_type": "properties",
"_id": "25467834901804247006200168495554902214",
"_version": 4,
"_score": 1,
"_source": {
"app_package_id": 4495665825523018,
"device_id": "b94b29c3-f03f-4e43-a646-53708e025779",
"group_id": 876,
"customer_user_id": "",
"customer_device_id": "b2f5fbfb-5d05-01e9-32e9-a8b332e9a8b3",
"advertise_id": "",
"device_os": "android",
"device_os_comparable_version": "00000004.00000002.00000002.00000017",
"device_os_full_name": "android 4.2.2.17",
"android_id": "c93334e4e246b83c",
"manufacturer": "OPPO",
"model": "R1001",
"screen_width": 480,
"screen_height": 800,
"device_language": "vi",
"cpu": "",
"is_rooted": 1,
"is_jailbroken": 0,
"vendor": "perion",
"app_installation_time": 1435173851000,
"push_allowed": 1,
"operator": "Beeline VN",
"mcc": "452",
"mnc": "07",
"mac_address": "",
"created_at": 1435174685000,
"registered_in_desktop": "",
"short_country_code": "VN",
"updated_at": 1435259712000,
"app_package_name": "com.gingersoftware.android.keyboard",
"app_version_type": ""
}
}
"manufacturer": "OPPO",
{"_index": "events-2015-06-04",
"_type": "events",
"_id": "AU4-zg5nOG4dkiMGKrCx",
"_version": 1,
"_score": 1,
"_source": {
"numeric_value_unit": "key",
"event_type_name": "custom",
"text_value": "",
"event_date": 1433440149000,
"numeric_value": 13,
"quantity": 0,
"event_name": "Saved Tap",
"numeric_value_name": "taps saved",
"app_package_id": 1433440149000,
"api_key": “asasa1w121",
"device_id": "c5eb1fe0-8a77-41ef-9f79-ed7ed69d32e6"
}
}
"event_name": "Saved Tap",
"event_name": "SVD T",
"event_name": "Saved TP",
"event_name": "Saved",

9. Constraints
• No BIG Brother
• Data can't be verified once it leaves
the application

10. Transactions
• Atomicity
• Locking
• Rollback

11. Relations type
• One to One
• One to Many
• Many to Many

12. Relations example
1 to Many
City : Person
1 to 1
Employee: Resume
{
_id:101,
Name:Jason Voorhees,
Age: 99
Resume_ID:1004
}
{
_id:1004,
Jobs:[Cook]
Education:[Knifery]
Hobbies:[Murder]
Employee_id:101
}
{
"name" : "Dam Square, Amsterdam",
"location" : {
"type" : "polygon",
"coordinates" : [[
[ 4.89218, 52.37356 ],
[ 4.89205, 52.37276 ], ….…
]]}
}
{
_id:101,
Name:Jason Voorhees,
Age: 99
Resume_ID:1004
}
Many to Many
Student : Teacher
{
_id:101,
Name:Jason Voorhees,
Age: 99
Resume_ID:1004
Courses:[“Chainsaw 101”,”Axing”],
Teachers:[101]
}
{
_id:101,
Name: “Freddy Krueger”,
Age: 60
Resume_ID:1004
Skills: [{Skill:}]
}

13. Embedding
• Embed
• Known doc size
• Data is highly related
• No joins
• Don’t Embed
• Very large data sets
• Data is updated rapidly

14. Doesn’t fit ….
• Use the data model that most fits your needs
• Don’t be afraid of Polyglot Persistence

15. Polyglot Persistence
• Data usage patterns
• Readers vs. Writers
• Online vs. Batch
• Concurrency
• Issues
• Data freshness
• Data consistency
• System Coupling

16. Questions?