The world is changing
Agenda
- New challenges and ways
- On-premises or cloud (or both?)
- The union of all
- Making sense of it
- How to get there...
Today, 80% of
organizations
adopt cloud-first
strategies
AI investment
increased by
300% in 2017
Data will grow to
44 ZB in 2020
Today, 80% of
organizations
adopt cloud-first
strategies
AI investment
increased by
300% in 2017
Data will grow to
44 ZB in 2020 C LO U D A IDATA
C LO U D
DATA A I
Organizations that harness data,
cloud, and AI outperform
Rely on a modern data estate
Patrik Borosch
TSP / DP
• 02.07.1971/married/Daughter(19)
• Music(Squared Circle/The Midcrise Liars)/climbing/skiing/cycling/Tango
• EDV-Kaufmann (german IHK 1995) = Data Processing with Cobol and Databases = the early BI
guys ;)
• Reporting and data processing in controlling departments
• BI Consultant/Senior Consultant: ASTECH Solutions/T-Systems/Trivadis/Avanade
•  Discipline Manager Microsoft BI and Power Pivot Trainer @ Trivadis
• Head of BI: Allianz Global Assistance
• TSP DP: Microsoft
• SQL Server/SSIS/SSAS/SSRS/MDM
• PowerBI/PowerPivot/PowerQuery
• Azure SQL DB/DW/Data Lake/Azure Stream Analytics/Data Factory/AAS
• SQL/DAX/MDX/(PowerShell)/(C#)
• Informatica/Microstrategy/Essbase/Enterprise Architect/UML/Perl/Unix/Linux  had that... been
there...
• First «Big Data»-Project in 2006: Teradata/Informatica/Microstrategy, 1.4TB = eight weeks for init
load = lots of fun :D
8
9
Dr. John Snow (1854)
One of the first visual
investigations of collected
data helped to solve a cholera
epidemic in Soho…
- You need the facts
- BUT you also need to
make sense of it...
Therefore you need to have
the right tools and methods...
The many
sources and
rapid growth
of data
requires a
new approach
• Sentiment
Analysis
• Social Media /
Sales
Connection
• Customer
Segmentation
Data lake
From Wikipedia, the free encyclopedia
A data lake is a method of storing data within a system or repository, in its natural format,[1] that
facilitates the collocation of data in various schemata and structural forms, usually object blobs or files.
The idea of data lake is to have a single store of all data in the enterprise ranging from raw data (which
implies exact copy of source system data) to transformed data which is used for various tasks including
reporting, visualization, analytics and machine learning. The data lake includes structured data from
relational databases (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured
data (emails, documents, PDFs) and even binary data (images, audio, video) thus creating a centralized
data store accommodating all forms of data.
James Dixon / Pentaho (2010)
Data Lakes
BUSINESS APPS
CUSTOM APPS
ANALYTICAL DASHBOARDS
AZURE SQL DATA WAREHOUSE
AZURE CLI
AZURE DATA FACTORY
BCP COMMAND LINE UTILITY
SQL SERVER INTEGRATION SERVICES
AZURE ANALYSIS SERVICES
BUSINESS APPS
CUSTOM APPS
ANALYTICAL DASHBOARDS
DATA FACTORY
ANALYTICAL DASHBOARDS
Polybase
AZURE SQL DATA WAREHOUSE
DATA FACTORY
AZURE ANALYSIS SERVICES
AZURE MACHINE LEARNING
& MACHINE LEARNING SERVER
AZURE COSMOS DB
AZURE STORAGE
BUSINESS APPS
CUSTOM APPS
ANALYTICAL DASHBOARDS
DATA FACTORY
ANALYTICAL DASHBOARDS
Polybase
AZURE SQL DATA WAREHOUSE
DATA FACTORY
AZURE ANALYSIS SERVICES
AZURE MACHINE LEARNING
& MACHINE LEARNING SERVER
AZURE COSMOS DB
AZURE HDINSIGHT
(Hadoop)AZURE STORAGE
BUSINESS APPS
CUSTOM APPS
ANALYTICAL DASHBOARDS
DATA FACTORY
ANALYTICAL DASHBOARDS
Polybase
AZURE SQL DATA WAREHOUSE
DATA FACTORY
AZURE ANALYSIS SERVICES
AZURE MACHINE LEARNING
& MACHINE LEARNING SERVER
AZURE COSMOS DB
AZURE STORAGE AZURE DATABRICKS
(SPARK)
BUSINESS APPS
CUSTOM APPS
ANALYTICAL DASHBOARDS
DATA FACTORY
AZURE DATA LAKE STORE AZURE DATA LAKE ANALYTICS
ANALYTICAL DASHBOARDS
Polybase
AZURE SQL DATA WAREHOUSE
DATA FACTORY
AZURE ANALYSIS SERVICES
AZURE MACHINE LEARNING
& MACHINE LEARNING SERVER
AZURE COSMOS DB
CONTROL EASE OF USE
Azure Data Lake
Analytics
Azure Data Lake Store
Azure Storage
Any Hadoop technology,
any distribution
Workload optimized,
managed clusters
Data Engineering in a
Job-as-a-service model
Azure Marketplace
HDP | CDH | MapR
Azure Data Lake
Analytics
IaaS Clusters Managed Clusters Big Data as-a-service
Azure HDInsight
Frictionless & Optimized
Spark clusters
Azure Databricks
BIGDATA
STORAGE
BIGDATA
ANALYTICS
ReducedAdministration
K N O W I N G T H E V A R I O U S B I G D A T A S O L U T I O N S
Drag & Drop
Azure ML
Big Data is driving transformative changes
Cost
Culture
Data
Characteristics
Traditional Big Data
Relational
(with highly modeled schema)
All Data
(with schema agility)
Expensive
(storage and compute capacity)
Cloud
(storage and compute capacity)
Rear-view reporting
(using relational algebra)
Intelligent action
(using relational algebra AND ML,
graph, streaming, image processing)
Cognitive Services
• Faces, images, emotion recognition and video intelligence
• Spoken language processing, speaker recognition, custom speech recognition
• Natural language processing, sentiment and topics analysis, spelling errors
• Complex tasks processing, knowledge exploration,
intelligent recommendations
• Bing engine capabilities for Web, Autosuggest, Image,
Video and News
Intelligence
Cortana
Bot
Framework
Cognitive
Services
Microsoft BI, the agile way…
Azure
Analysis Services
Data Sources Ingest Prepare Analyze Publish Consume
Sensors
and devices
Stream
Analytics
Diagnostic
Streaming
Power BI
Sources
- Oralce HFS
- SAP BW
- …
Azure Data Lake Store
Data Factory: Move data, orchestrate, schedule and monitor
Azure Data LakeIoT Hubs
Machine
Learning
HDInsight
Data Science
Workbench
Stream
Analytics
Power BI Report Server
Architecture Blueprint
SSIS
SQL Server 2017: Security, Performance, Polybase, ML Services, Analytics
SQL Server
2017
SSAS
BI Bot
Apps
Lab- and
other Apps
AzureDataPlatformSQLserver2017
AI built-in | Most secure | Lowest TCO
M I C R O S O F T F O R Y O U R M O D E R N D A T A E S T A T E
Data warehouses
Data lakes
Operational databases
Data warehouses
Data lakes
Operational databases
SQL Server Azure Data Services
Industry leader 4 years in a row
#1 TPC-H performance
T-SQL query over any data
70% faster
2x the global reach
99.9% SLA
HYBRID
Easiest lift and shift
with no code changes
SocialLOB Graph IoTImageCRM
Security and
performance
Flexibility
of choice
Reason over
any data, anywhere
Tools for your
migration journey
SQL Server Migration Assistant (SSMA)
Automates database migration to SQL Server from
Microsoft Access, DB2, MySQL, Oracle, and SAP ASE.
Data Migration Assistant (DMA)
Enables upgrade to SQL Server and Azure SQL
Database.
Database Experimentation Assistant (DEA)
Assists in evaluating a targeted version of SQL for
a given workload.
Azure Hybrid Benefit for SQL Server
Maximizes current on-premises license investments
to facilitate migration.
Azure SQL Database Managed Instance
Facilitates lift and shift migration from
on-premises SQL Server to PaaS.
Azure Database Migration Service (Azure DMS)
SQL Server Migration Assistant (SSMA)
Data Migration Assistant (DMA)
Database Experimentation Assistant (DEA)
SQL Database
Managed Instance
Azure Hybrid
Benefit for
SQL Server
Empower today’s innovators to unleash the power of data
and reimagine possibilities that will improve our world

Trivadis Azure Data Lake

  • 1.
    The world ischanging
  • 2.
    Agenda - New challengesand ways - On-premises or cloud (or both?) - The union of all - Making sense of it - How to get there...
  • 3.
    Today, 80% of organizations adoptcloud-first strategies AI investment increased by 300% in 2017 Data will grow to 44 ZB in 2020
  • 4.
    Today, 80% of organizations adoptcloud-first strategies AI investment increased by 300% in 2017 Data will grow to 44 ZB in 2020 C LO U D A IDATA
  • 5.
    C LO UD DATA A I Organizations that harness data, cloud, and AI outperform
  • 6.
    Rely on amodern data estate
  • 7.
    Patrik Borosch TSP /DP • 02.07.1971/married/Daughter(19) • Music(Squared Circle/The Midcrise Liars)/climbing/skiing/cycling/Tango • EDV-Kaufmann (german IHK 1995) = Data Processing with Cobol and Databases = the early BI guys ;) • Reporting and data processing in controlling departments • BI Consultant/Senior Consultant: ASTECH Solutions/T-Systems/Trivadis/Avanade •  Discipline Manager Microsoft BI and Power Pivot Trainer @ Trivadis • Head of BI: Allianz Global Assistance • TSP DP: Microsoft • SQL Server/SSIS/SSAS/SSRS/MDM • PowerBI/PowerPivot/PowerQuery • Azure SQL DB/DW/Data Lake/Azure Stream Analytics/Data Factory/AAS • SQL/DAX/MDX/(PowerShell)/(C#) • Informatica/Microstrategy/Essbase/Enterprise Architect/UML/Perl/Unix/Linux  had that... been there... • First «Big Data»-Project in 2006: Teradata/Informatica/Microstrategy, 1.4TB = eight weeks for init load = lots of fun :D
  • 8.
  • 9.
  • 10.
    Dr. John Snow(1854) One of the first visual investigations of collected data helped to solve a cholera epidemic in Soho… - You need the facts - BUT you also need to make sense of it... Therefore you need to have the right tools and methods...
  • 11.
    The many sources and rapidgrowth of data requires a new approach • Sentiment Analysis • Social Media / Sales Connection • Customer Segmentation
  • 12.
    Data lake From Wikipedia,the free encyclopedia A data lake is a method of storing data within a system or repository, in its natural format,[1] that facilitates the collocation of data in various schemata and structural forms, usually object blobs or files. The idea of data lake is to have a single store of all data in the enterprise ranging from raw data (which implies exact copy of source system data) to transformed data which is used for various tasks including reporting, visualization, analytics and machine learning. The data lake includes structured data from relational databases (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured data (emails, documents, PDFs) and even binary data (images, audio, video) thus creating a centralized data store accommodating all forms of data. James Dixon / Pentaho (2010) Data Lakes
  • 13.
    BUSINESS APPS CUSTOM APPS ANALYTICALDASHBOARDS AZURE SQL DATA WAREHOUSE AZURE CLI AZURE DATA FACTORY BCP COMMAND LINE UTILITY SQL SERVER INTEGRATION SERVICES AZURE ANALYSIS SERVICES
  • 14.
    BUSINESS APPS CUSTOM APPS ANALYTICALDASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE STORAGE
  • 15.
    BUSINESS APPS CUSTOM APPS ANALYTICALDASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE HDINSIGHT (Hadoop)AZURE STORAGE
  • 16.
    BUSINESS APPS CUSTOM APPS ANALYTICALDASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE STORAGE AZURE DATABRICKS (SPARK)
  • 17.
    BUSINESS APPS CUSTOM APPS ANALYTICALDASHBOARDS DATA FACTORY AZURE DATA LAKE STORE AZURE DATA LAKE ANALYTICS ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB
  • 19.
    CONTROL EASE OFUSE Azure Data Lake Analytics Azure Data Lake Store Azure Storage Any Hadoop technology, any distribution Workload optimized, managed clusters Data Engineering in a Job-as-a-service model Azure Marketplace HDP | CDH | MapR Azure Data Lake Analytics IaaS Clusters Managed Clusters Big Data as-a-service Azure HDInsight Frictionless & Optimized Spark clusters Azure Databricks BIGDATA STORAGE BIGDATA ANALYTICS ReducedAdministration K N O W I N G T H E V A R I O U S B I G D A T A S O L U T I O N S Drag & Drop Azure ML
  • 20.
    Big Data isdriving transformative changes Cost Culture Data Characteristics Traditional Big Data Relational (with highly modeled schema) All Data (with schema agility) Expensive (storage and compute capacity) Cloud (storage and compute capacity) Rear-view reporting (using relational algebra) Intelligent action (using relational algebra AND ML, graph, streaming, image processing)
  • 21.
    Cognitive Services • Faces,images, emotion recognition and video intelligence • Spoken language processing, speaker recognition, custom speech recognition • Natural language processing, sentiment and topics analysis, spelling errors • Complex tasks processing, knowledge exploration, intelligent recommendations • Bing engine capabilities for Web, Autosuggest, Image, Video and News Intelligence Cortana Bot Framework Cognitive Services
  • 22.
    Microsoft BI, theagile way… Azure Analysis Services
  • 24.
    Data Sources IngestPrepare Analyze Publish Consume Sensors and devices Stream Analytics Diagnostic Streaming Power BI Sources - Oralce HFS - SAP BW - … Azure Data Lake Store Data Factory: Move data, orchestrate, schedule and monitor Azure Data LakeIoT Hubs Machine Learning HDInsight Data Science Workbench Stream Analytics Power BI Report Server Architecture Blueprint SSIS SQL Server 2017: Security, Performance, Polybase, ML Services, Analytics SQL Server 2017 SSAS BI Bot Apps Lab- and other Apps AzureDataPlatformSQLserver2017
  • 25.
    AI built-in |Most secure | Lowest TCO M I C R O S O F T F O R Y O U R M O D E R N D A T A E S T A T E Data warehouses Data lakes Operational databases Data warehouses Data lakes Operational databases SQL Server Azure Data Services Industry leader 4 years in a row #1 TPC-H performance T-SQL query over any data 70% faster 2x the global reach 99.9% SLA HYBRID Easiest lift and shift with no code changes SocialLOB Graph IoTImageCRM Security and performance Flexibility of choice Reason over any data, anywhere
  • 26.
    Tools for your migrationjourney SQL Server Migration Assistant (SSMA) Automates database migration to SQL Server from Microsoft Access, DB2, MySQL, Oracle, and SAP ASE. Data Migration Assistant (DMA) Enables upgrade to SQL Server and Azure SQL Database. Database Experimentation Assistant (DEA) Assists in evaluating a targeted version of SQL for a given workload. Azure Hybrid Benefit for SQL Server Maximizes current on-premises license investments to facilitate migration. Azure SQL Database Managed Instance Facilitates lift and shift migration from on-premises SQL Server to PaaS. Azure Database Migration Service (Azure DMS) SQL Server Migration Assistant (SSMA) Data Migration Assistant (DMA) Database Experimentation Assistant (DEA) SQL Database Managed Instance Azure Hybrid Benefit for SQL Server
  • 27.
    Empower today’s innovatorsto unleash the power of data and reimagine possibilities that will improve our world