Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Azure Data Lake Store and Analytics

224 views

Published on

With Azure Data Lake Store, analyze all of your data in one place with no artificial constraints. Data Lake Store can store trillions of files.
Azure Data Lake Analytics: Easily develop and run massively parallel data transformation and processing programs in U-SQL, R, Python, and .NET over petabytes of data. With no infrastructure to manage, you can process data on demand, scale instantly, and only pay per job.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Azure Data Lake Store and Analytics

  1. 1. Azure Data Lake Store & Azure Data Lake Analytics Sergio Zenatti Filho, Associate Director Data & Analytics, Satalyst
  2. 2. Sergio Zenatti Filho Associate Director Data &Analytics - Satalyst I am Data and Analytics Director with over 16 years experience in the delivery of Business Intelligence and Analytics Solutions. I worked internationally around Australia, New Zealand and Brazil, in sectors that include Mining, Oil & Gas, Government, Healthcare, Financial Services, Telecom, Automotive and dairy. I enjoy learning new technology and help people to learn. Place your photo here /sergiozenatti @SergioZenatti zenatti.net
  3. 3. SQL Saturday Perth - 2018 http://www.sqlsaturday.com/761
  4. 4. Session objectives and key takeaways
  5. 5. What is Data Lake? Ingest all data regardless of requirements Store all data in native format without schema definition Do analysis Hadoop, Spark, R, Azure Data Lake Analytics (ADLA) Interactive queries Batch queries Machine Learning Data warehouse Devices
  6. 6. The 3 Azure Data Lake Services
  7. 7. Azure Data Lake (ADL) Store • A hyper-scale repository for Big Data analytics workloads; • Hadoop File System (HDFS) for the cloud; • Unlimited storage and can host petabyte files; • Store any data in its native format; • Enterprise-grade access control and encryption;
  8. 8. Data Lake Store
  9. 9. DEMO Provision Azure Data Lake Store
  10. 10. Azure Data Lake Analytics • An on-demand analytics job service in the cloud; • Run massively parallel data transformation and processing programs in U-SQL, R, Python, and .NET; • No infrastructure to manage, you can process data on demand, scale instantly, and only pay per job; • Integrates with Visual Studio to develop, debug and tune code faster; Azure Data Lake Analytics Unit (AU): is a unit of computation made available to your U-SQL job. Each AU gives your job access to a set of underlying resources such as CPU and memory.
  11. 11. ADLAnalytics – Query U-SQL Query Query Query Query W rite Azure Storage Blobs Azure SQL in VMs Azure SQL DB Azure Data Lake Analytics Query Azure SQL Data Warehouse Query Write Azure Data Lake Storage
  12. 12. U-SQL • It’s a framework for Big Data; • Familiar syntax to millions of SQL & .NET developers; • Built on the same distributed runtime that powers the big data systems inside Microsoft; • Querying multiple Azure Data Sources (Federated Query);
  13. 13. Cognitive Capabilities in U-SQL • Image Tagging • Emotion Extraction • Face Detection • Optical Character Recognition • Key Phrases Extraction • Sentiment Analysis
  14. 14. DEMO Provision Azure Data Lake Analytics U-SQL: Face Detection New York Taxi Data
  15. 15. What next? • https://mva.microsoft.com/en-us/training-courses/data-series-analytics- big-data-azure-data-lake-17759 • https://www.edx.org/course/processing-big-data-with-azure-data-lake- analytics • https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake- analytics-data-lake-tools-get-started
  16. 16. Thank you for your time! Sergio Zenatti Filho Associate Director Data & Analytics, Satalyst sergiozenatti @SergioZenatti zenatti.net

×