Azure Data Lake Store and Analytics

1. Harness the power of big data
using Azure Data Lake
Sergio Zenatti Filho

2. Thanks to our Gold Sponsors: SQL Sat Auckland
Sergio Zenatti Filho
Associate Director Data & Analytics - Satalyst
I am Data and Analytics Director with over 16 years
experience in the delivery of Business Intelligence
and Analytics Solutions. I worked internationally
around Australia, New Zealand and Brazil, in sectors
that include Mining, Oil & Gas, Government,
Healthcare, Financial Services, Telecom, Automotive
and dairy. I enjoy learning new technology and help
people to learn.
Place your
photo here
/sergiozenatti @SergioZenatti zenatti.net

3. Thanks to our Gold Sponsors: SQL Sat Auckland
Thanks to all sponsors

4. Thanks to our Gold Sponsors: SQL Sat Auckland
Session Objectives
• What is Data Lake
• Azure Data Lake Store
• Azure Data Lake Analytics
• U-SQL
• Demo

5. Thanks to our Gold Sponsors: SQL Sat Auckland
What is Data Lake
Ingest all data
regardless of requirements
Store all data
in native format
without schema
definition
Do analysis
Hadoop, Spark, R,
Azure Data Lake
Analytics (ADLA)
Interactive queries
Batch queries
Machine Learning
Data warehouse
Devices

6. Thanks to our Gold Sponsors: SQL Sat Auckland
The 3 Azure Data Lake Services

7. Thanks to our Gold Sponsors: SQL Sat Auckland
Azure Data Lake Store
• A hyper-scale repository for Big Data
analytics workloads;
• Hadoop File System (HDFS) for the
cloud;
• Unlimited storage and can host petabyte files;
• Store any data in its native format;
• Enterprise-grade access control and
encryption;

8. Thanks to our Gold Sponsors: SQL Sat Auckland
Azure Data Lake Store

9. DEMO
Data Lake Store

10. Thanks to our Gold Sponsors: SQL Sat Auckland
Azure Data Lake Analytics
• An on-demand analytics job service in the cloud;
• Run massively parallel data transformation and processing programs
in U-SQL, R, Python, and .NET;
• No infrastructure to manage, you can process data on demand, scale
instantly, and only pay per job;
• Integrates with Visual Studio to develop, debug and tune code faster;
Azure Data Lake Analytics Unit (AU): is a unit of computation made
available to your U-SQL job. Each AU gives your job access to a set of
underlying resources such as CPU and memory.

11. Thanks to our Gold Sponsors: SQL Sat Auckland
Azure Data Lake Analytics - Query
U-SQL
Query
Query
Query
Query
W
rite
Azure
Storage Blobs
SQL
in VMs
Azure
SQL DB
Azure Data
Lake Analytics
Query
Azure
SQL Data Warehouse
Query
Write
Azure
Data Lake Storage

12. Thanks to our Gold Sponsors: SQL Sat Auckland
U-SQL
• It’s a framework for Big Data;
• Familiar syntax to millions of
SQL & .NET developers;
• Built on the same distributed
runtime that powers the big
data systems inside Microsoft;
• Querying multiple Azure Data
Sources (Federated Query);

13. Thanks to our Gold Sponsors: SQL Sat Auckland
Cognitive Capabilities in U-SQL
• Image Tagging
• Emotion Extraction
• Face Detection
• Optical Character Recognition
• Key Phrases Extraction
• Sentiment Analysis

14. DEMO
Data Lake Analytics

15. Thanks to our Gold Sponsors: SQL Sat Auckland
ADLA – Real Scenario
• Process around 15k XML files a day with total size around 6-7 GB
• Infrastructure cost: max $5k a month
• Process files once every hour
• ADF, Logic Apps, Azure Data Lake, Azure SQL DW and Azure Analysis Services

16. Thanks to our Gold Sponsors: SQL Sat Auckland
What Next?
• https://mva.microsoft.com/en-us/training-courses/data-series-analytics-
big-data-azure-data-lake-17759
• https://www.edx.org/course/processing-big-data-with-azure-data-lake-
analytics
• https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-
analytics-data-lake-tools-get-started

17. Thanks to our Gold Sponsors: SQL Sat Auckland
Q&A

18. Thank you
for your time!
Sergio Zenatti Filho
Associate Director Data & Analytics,
Satalyst
sergiozenatti @SergioZenatti zenatti.net