The session will show how to do Analyze and visualize non-relational data with DocumentDB + Power BI. We are in the midst of a paradigm shift on how we store and analyze data. Unstructured or flexible schema data represents a large portion of data within an organization. Everyone is obsessed to turn this data into meaningful business information. Unstructured data analytics do not need to be time consuming and complex. Come learn how to analyze and visualize unstructured data in DocumentDB.

1. Analyze and visualize non-relational data
with Azure DocumentDB +
HDInsight + Power BI
Han Wong
Program Manager
Azure DocumentDB

2. Session Objectives And Takeaways
Internet
of Things
Actionable
Data Insights
Scalable
Solutions

3. Internet
of Things
Actionable
Data Insights
Scalable
Solutions

4. NoSQL - document databases

5. Microsoft Confidential
Modern Applications…
Are developed iteratively with
continuously evolving data models
Produce and consume data at
unprecedented rates
Require instantaneous response times
to match user expectations
Experience unpredictable and
explosive growth

6. Microsoft Confidential
Developers want…

7. • Part of NoSQL family
• Built for simplicity, scale and performance
• Non-relational, no enforced schema
• Fully managed
Document Database Services

8. Not these
documents

9. {
“id":“itemdata2344",
“data":“TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpbmd1aXNoZWQsIG5vdCBvb
m
cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIH
nJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmlt
dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2
bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2
cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIH
nJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmlt
dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2
bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2
cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIH
nJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmlt
dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2
bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2
cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIH
nJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmlt
dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2
bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2
cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIH
nJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmlt
dGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
Not ideal for these
documents

10. {
"name": "SmugMug",
"permalink": "smugmug",
"homepage_url": "http://www.smugmug.com",
"blog_url": "http://blogs.smugmug.com/",
"category_code": "photo_video",
"products": [
{
"name": "SmugMug",
"permalink": "smugmug"
}
],
"offices": [
{
"description": "",
"address1": "67 E. Evelyn Ave",
"address2": "",
"zip_code": "94041",
"city": "Mountain View",
"state_code": "CA",
"country_code": "USA",
"latitude": 37.390056,
"longitude": -122.067692
}
]
}
Perfect for these
Documents
schema-agnostic JSON store
for
hierarchical and de-normalized data at scale

11. • Rich querying over flexible schemas
• Automatic indexing of every document property
• Easy-to-use SQL grammar
• Integrated with other Azure data services
Azure DocumentDB

12. Catalog Data
Preferences
&
State Data
Gaming
Events
&
Logging
User Data
&
Content
Data
Exchange
Common Use Cases
Next Games
Halo5

13. The basics

14. The Basics
• DocumentDB Resources

15. The Basics
• DocumentDB Resources

16. The Basics
• DocumentDB Resources
!=

17. The Basics
• DocumentDB Resources

18. The Basics
• DocumentDB Resources

19. Performance levels

20. Performance Levels

21. Performance Levels
S1 S2 S3

22. Performance Levels
S1 S2 S3

23. Performance Levels
S2 S3S3
http://aka.ms/docdbperf

24. Scaling through Collections

25. Scaling through Collections

26. Demo - Portal & Tools

27. The Scenario

28. Running a Database as a Service
CPU Utilization Storage Utilization

29. Sample JSON data
{
"TIMESTAMP": "12/10/2015 00:00:00",
"TimeStampEpoch": 1449705600,
"Server": "ProdApp002",
"Counter" : "percent processor
time",
"UnitMeasure" : "percent",
"CounterValue" : 4.436914
}
CPU Utilization Storage Utilization
{
"TIMESTAMP": "12/10/2015 00:10:00",
"TimeStampEpoch": 1449706200
"Server": "ProdApp002",
"Counter" : "percent free space",
"UnitMeasure" : "percent",
"Location" : "d",
"Type" : "data",
"CounterValue" : 96.717065
}

30. Goal – Pinpoint DocumentDB Hotspots
CPU Utilization Storage Utilization

31. Solution – Power BI Integration
{ }
SELECT * FROM
analytics
34.56%
"TIMESTAMP": "2015-12-10T00:00:00Z"
"Server": "ProdApp002"
"Counter" : "percent processor time"
"UnitMeasure" : "percent"
"CounterValue" : 4.436914
"TIMESTAMP": "2015-12-10T00:10:00Z"
"Server": "ProdApp002"
"Counter" : "percent free space"
"UnitMeasure" : "percent"
"CounterValue" : 96.717065
12.09%

33. Demo

34. Goal – Generate 50th, 90th, and 99th percentiles
CPU Utilization Storage Utilization

35. Solution – HDInsight Integration
{ }
SELECT * FROM
logs201512 WHERE
logs201512.Counter =
“percent processor time”
SELECT * FROM
logs201512 WHERE
logs201512.Counter =
“percent free space”
percentile(CounterValue, 0.50)
percentile(CounterValue, 0.90)
percentile(CounterValue, 0.99)

36. Solution – Full Solution
{ }
SELECT * FROM
analytics
percentile(CounterValue, 0.50)
percentile(CounterValue, 0.90)
percentile(CounterValue, 0.99)
SELECT * FROM
logs201512_summary
SELECT * FROM
logs201512
34.56
%
"TIMESTAMP": "2015-12-10T00:00:00Z"
"Server": "ProdApp002"
"Counter" : "percent processor time"
"UnitMeasure" : "percent"
"CounterValue" : 4.436914
"TIMESTAMP": "2015-12-10T00:10:00Z"
"Server": "ProdApp002"
"Counter" : "percent free space"
"UnitMeasure" : "percent"
"CounterValue" : 96.717065
12.09% P50
28%
P90
36%
P99
54%

38. Overall
Great fit for Logging and Event based scenarios
 Adhoc, real-time queries
 Integration with other Azure Data services:
HDInsight, Stream Analytics, Data Factory, and Power BI
Great fit for dynamic, evolving applications
 On the fly schema changes
 Automatic indexing
 Horizontal Scaling

39. Get Started Today
select * from
playground p where
p.name = “DocumentDB”
http://aka.ms/docdbplayground http://aka.ms/docdbstarter http://aka.ms/docdbimport