Presented by SlamData CTO John De Goes at the Percona Live Data Performance Conference on April 20, 2016.

1. @slamdata @jdegoes
John A. De Goes — CTO SlamData Inc.
Give Me My Damn
Report: Making NoSQL
Data Accessible to the
Business

2. @slamdata @jdegoes
Agenda
1. The Rise of NoSQL
2. The Dark Side of NoSQL
3. Options for Reporting
a. Extract-Transform-Load
b. Fat Drivers
c. Code to NoSQL APIs
d. Native NoSQL Analytics
4. Why NoSQL Analytics is Hard
5. NoSQL Databases: Not Equal
6. Question & Answer

3. @slamdata @jdegoes
The Rise of NoSQL

4. @slamdata @jdegoes
The Rise of NoSQL

5. @slamdata @jdegoes
The Rise of NoSQL
● Massively scalable
● Operational Ease-of-Use
● Native support for rich data structures
● Native Support for heterogeneity
● Rapid Time-to-Deployment

6. @slamdata @jdegoes
The Rise of NoSQL

7. @slamdata @jdegoes
The Dark Side of
NoSQL

8. @slamdata @jdegoes
The Dark Side of NoSQL
Overview

9. @slamdata @jdegoes
The Dark Side of NoSQL
Give Me My Damn Report!
● Ad hoc analytics
● Exploratory analytics
● Operational analytics
● Analytics dashboards
● Batch reporting
● IoT / Event analytics
Need for Analytics

10. @slamdata @jdegoes
The Dark Side of NoSQL
SQL Analytics

11. @slamdata @jdegoes
The Dark Side of NoSQL
1. ETL
2. Fat Drivers
3. Code to NoSQL API
4. Native NoSQL ANalytics
Choices

12. @slamdata @jdegoes
Options for
Reporting

13. @slamdata @jdegoes
Extract-Transform-Load
{"user_id": "mary@bigco.com",
"profile": {
"name": "Mary Jane",
"addresses": [{
"city": "London",
"country": "UK"
}],
"band_plays": {
"Squirrel Nut Zippers": 56,
"Red Hot Tomatoes": 19,
"Big Bad Voodoo Daddy": 102
}
}
SQL /
Hadoop
Overview

14. @slamdata @jdegoes
Extract-Transform-Load
1. Flattening
users
user_id
mary@bigco.com
...
...
band_plays
user_id band_name play_count
mary@bigco.com Squirrel Nut Zippers 56
mary@bigco.com Red Hot Tomatoes 19
mary@bigco.com Big Bad Voodoo Daddies 102
profiles
profile_id user_id name
1 mary@bigco.com Mary Jane
addresses
profile_id city country
1 London UK

15. @slamdata @jdegoes
Extract-Transform-Load
2. Homogenization
events
type user_id genre_name artist_name band_name play_count
“band_play” ... NULL NULL “Squirrel Nut Zippers” 56
“artist_play” ... NULL “Frank Sinatra” NULL 19
“genre_play” ... “New Age” NULL NULL 102

16. @slamdata @jdegoes
Extract-Transform-Load
3. Incremental ETL
1. Last_modified Field
2. Import changed data*
* Less relevant for Hadoop

17. @slamdata @jdegoes
Extract-Transform-Load
Tools

18. @slamdata @jdegoes
Extract-Transform-Load
Report Card
✗ Slow
✗ Painful
✗ Brittle
✓ Tunable Performance
✓ Unlimited Flexibility in
Reporting / Analytics

19. @slamdata @jdegoes
Fat Drivers
Overview
Driver
Embedded SQL Engine
Real-Time ETL
(Filtered Table Scan)

20. @slamdata @jdegoes
Fat Drivers
Approaches
Magic Config

21. @slamdata @jdegoes
Fat Drivers
Vendors

22. @slamdata @jdegoes
Fat Drivers
Report Card
✗ Slow
✗ Limited to Small Data
✗ Limited to Simple Analytics
✗ Limited to Simple Data
✓ Low Friction
✓ Flexibility in Analytics /
Reporting

23. @slamdata @jdegoes
Code to NoSQL API
Overview
Code
CSV
HTML5/Javascript

24. @slamdata @jdegoes
Code to NoSQL API
Report Card
✗ Slow
✗ Painful
✗ Brittle
✗ Performance
✓ No ETL

25. @slamdata @jdegoes
Native NoSQL Analytics
Overview
Native NoSQL Analytics

26. @slamdata @jdegoes
Native NoSQL Analytics
Tools
SQL (+/-)
Visual Analytics
ETL (+/-) Native
ZoomData
Cloud 9 Charts
JSON Studio
Apache Drill
Quasar
SlamData
Impala

27. @slamdata @jdegoes
Native NoSQL Analytics
Report Card
✗ Immature
✗ Learning Curve
✗ Limited Choices
✓ No ETL
✓ Flexible & Fast
✓ Any data, Anywhere
✓ Tunable Performance

28. @slamdata @jdegoes
Why NoSQL Analytics
Is Hard

29. @slamdata @jdegoes
The
Eight
Deadly
Obstacles
to NoSQL Analytics

30. @slamdata @jdegoes
CHaracteristics
1. Generic Data Model

31. @slamdata @jdegoes
CHaracteristics
2 Isomorphic Data Model
Data SQL²
{
"userId": 8927524,
"profile": {
"name": "Mary Jane",
"age": 29,
"gender": "female"
},
"comments": [{
"id": "F2372BAC",
"text": "I concur.",
"replyTo": [9817361, "F8ACD164F"],
"time": "2015-02-03"
}, {
"id": "GH732AFC",
"replyTo": [9654726, "A44124F"],
"time": "2015-03-01"
}]
}
SELECT comments[*].replyTo[*]
FROM data

32. @slamdata @jdegoes
CHaracteristics
3. Multidimensionality
Data SQL²
{"user_id": 928347234,
"email": null,
"events": [
{"impression":{
"ts": 912348934,
"page": "index.html"}}]}
SELECT user_id,
[events[_] WHERE
events[_].ts < 9347234 ...]
AS events
FROM visitors

33. @slamdata @jdegoes
CHaracteristics
4. Unified Schema/Data
Data SQL²
{"user_id": "mary@bigco.com",
"band_plays":{
"Squirrel Nut Zippers": 56,
"Red Hot Tomatoes": 19,
"Big Bad Voodoo Daddy": 102}}
SELECT band_plays{*:} AS artistName,
SUM(band_plays{*}) AS votes
FROM music
GROUP BY band_plays{*:}

34. @slamdata @jdegoes
CHaracteristics
5. Polymorphic Queries
Data SQL²
{"type": "click",
"link": "http://foo.com"
"timestamp": 123987172}
{"type": "impression",
"page": "index.html"
"timestamp": 92372}
SELECT COUNT(*) AS count, timestamp
FROM data
GROUP BY timestamp

35. @slamdata @jdegoes
CHaracteristics
6. Post-Relational
Data SQL²
{"name": "John Doe",
"blog_posts": [
{"post_id": "89934"},
{"post_id": "92371"}
]}
SELECT authors.name, posts.title
FROM authors
JOIN posts
ON authors.blog_posts[*].post_id =
posts._id

36. @slamdata @jdegoes
CHaracteristics
7. Runtime Type Id & ConverSION
Data SQL²
{"email": ["joe@gmail.com",
"joe@slamdata.com"]}
{"email": {
"home": "sarah@gmail.com",
"work": "sarah@slamdata.com"}}
SELECT
CASE TYPEOF email
-- old: email stored in 2nd el:
WHEN 'array' THEN email[1]
-- new format:
WHEN 'map' THEN email.work
ELSE email
END AS email
FROM users

37. @slamdata @jdegoes
CHaracteristics
8. Structural Pattern Matching
Data SQL²
{"user_id": "toby@bing.com",
"events": [{"type": "purchase",
"timestamp": 12392342,
"order_id": "2ffa34aa"},
{"type": "click",
"timestamp": 92327123,
"link": "http://foo.com"}]}
SELECT
CASE user_events
WHEN […, e1, e2, …] THEN
e1.timestamp - e2.timestamp
END AS delta
FROM users

38. @slamdata @jdegoes
NoSQL Databases:
Not Equal

39. @slamdata @jdegoes
NoSQL Databases: Not Equal
Desired Characteristics
1. DUal Operations & Analytics
2. In-Database Analytics
3. General-Purpose Analytics
4. Native Report Tooling

40. @slamdata @jdegoes
NoSQL Databases: Not Equal
Couchbase
✓ Dual Operations / Analytics
✓ In-Database Analytics
✓ General-Purpose Analytics
✗ Native Report Tooling
Best Reporting Option: Fat Drivers
Runner-Up: Code to NoSQL APIs

41. @slamdata @jdegoes
NoSQL Databases: Not Equal
MarkLogic
✓ Dual Operations / Analytics
✓ In-Database Analytics
✓ General-Purpose Analytics
✗ Native Report Tooling
Best Reporting Option: ETL
Runner-Up: Code to NoSQL APIs

42. @slamdata @jdegoes
NoSQL Databases: Not Equal
MongoDB
✓ Dual Operations / Analytics*
✓ In-Database Analytics
✗ General-Purpose Analytics
✓ Native Report Tooling
Best Reporting Option: Native NoSQL Analytics
Runner-Up: Code to NoSQL APIs
* Further maturation needed

43. @slamdata @jdegoes
NoSQL Databases: Not Equal
ElasticSearch
✓ Dual Operations / Analytics
✓ In-Database Analytics
✗ General-Purpose Analytics
✗ Native Report Tooling
Best Reporting Option: Code to NoSQL APIs
Runner-Up: ETL to Hadoop

44. @slamdata @jdegoes
NoSQL Databases: Not Equal
Cassandra
✗ Dual Operations / Analytics*
✗ In-Database Analytics*
✗ General-Purpose Analytics
✗ Native Report Tooling
Best Reporting Option: ETL
Runner-Up: Code to NoSQL APIS*
* Real-time analytics

45. @slamdata @jdegoes
THE END
Questions?