Azure Data Warehouse
Overview
Justin Munsters
System Analyst - Holland
Microsoft Azure
Internet of Things
Audio / Video
Log Files
Text/Image
Social Sentiment
Data Market Feeds
eGov Feeds
Weather
Wikis / BlogsClick Stream Sensors / RFID / Devices
Spatial & GPS Coordinates
Modern WebMobile
Advertising CollaborationeCommerce
Digital Marketing
Search Marketing
Web Logs
Recommendations
ERP / CRM
Sales Pipeline
Payables
Payroll
Inventory
Contacts
Deal Tracking
Terabytes
(10e12)
Gigabytes
(1e9)
Petabytes
(10e15)
Velocity | Variety | Variability
Volume
ERP / CRM / ECM Modern Web Internet of Things
What is Big Data?
Data Warehouse Discussion
How big is this thing?
What about HA and redundancy?
How many cores, how much RAM?
How do we handle peaks and bursts?
Where is this thing going?
Clusters, Rackspace, Build Up/Out
Why the cloud?
Rapidly setup environments to drive business priorities
Scale to meet peak demands
Increase daily activities, efficiency and reduced cost.
Data Use & Computing Patterns
Services with micro seasonality trends
Peaks due to periodic increased demand
IT complexity and wasted capacity
Azure Portal UX
Redundancy & Reliability
Enterprise-ready SQL on a mature cloud platform for sophisticated DW scenarios
• Microsoft Azure Storage with replicas
• SQL Database built-in backup servers
• Failover with Microsoft Azure Traffic Manager
• Auto recovery of crashed/nonresponsive instances
• Fault domain to scatter instances across racks
• Swap to allow promoting staging to production
The Cost To Do This On-Premise? Azure
On-Premise Data Warehouse
?
• Hardware
• Software
• Data Center
• Off-Site Backup
• Staff
$$$$
$$$$
$$$$
Data Warehouse Unit (DWU)
Simply buy the query performance you need, not just hardware
Quantified by workload objectives: how fast rows are scanned, loaded, copied
Measure of Power
Transparency
First DW service to offer compute power on demand, independent of storageOn Demand
Scan 1B rows
100 DWU = 297 sec
400 DWU = 74 sec
800 DWU = 37 sec
1,600 DWU = 19 sec
Scan Rate xx M row/sec
Loading Rate xx K row/sec
Table Copy Rate xx K row/sec
100 DWU
Flexible Configuration & Costing
SQL Server Compatibility
Familiar SQL Server scripts and tools you have been using just work
Modular programming
(write once, execute multiple
times)
Faster code execution
Encapsulated
programming logic
Easier maintenance of
large tables
Improves performance
Enhanced scalability
and availability
Allows proper use and
comparisons of
characters in different
languages
Mature Column-Store
technology for best-
in-class DW query
performance
Complete, consistent, scalable data platform
Azure Data Warehouse

Microsoft Azure Data Warehouse Overview

  • 1.
    Azure Data Warehouse Overview JustinMunsters System Analyst - Holland Microsoft Azure
  • 2.
    Internet of Things Audio/ Video Log Files Text/Image Social Sentiment Data Market Feeds eGov Feeds Weather Wikis / BlogsClick Stream Sensors / RFID / Devices Spatial & GPS Coordinates Modern WebMobile Advertising CollaborationeCommerce Digital Marketing Search Marketing Web Logs Recommendations ERP / CRM Sales Pipeline Payables Payroll Inventory Contacts Deal Tracking Terabytes (10e12) Gigabytes (1e9) Petabytes (10e15) Velocity | Variety | Variability Volume ERP / CRM / ECM Modern Web Internet of Things What is Big Data?
  • 3.
    Data Warehouse Discussion Howbig is this thing? What about HA and redundancy? How many cores, how much RAM? How do we handle peaks and bursts? Where is this thing going? Clusters, Rackspace, Build Up/Out
  • 4.
    Why the cloud? Rapidlysetup environments to drive business priorities Scale to meet peak demands Increase daily activities, efficiency and reduced cost.
  • 5.
    Data Use &Computing Patterns Services with micro seasonality trends Peaks due to periodic increased demand IT complexity and wasted capacity
  • 6.
  • 7.
    Redundancy & Reliability Enterprise-readySQL on a mature cloud platform for sophisticated DW scenarios • Microsoft Azure Storage with replicas • SQL Database built-in backup servers • Failover with Microsoft Azure Traffic Manager • Auto recovery of crashed/nonresponsive instances • Fault domain to scatter instances across racks • Swap to allow promoting staging to production
  • 8.
    The Cost ToDo This On-Premise? Azure On-Premise Data Warehouse ? • Hardware • Software • Data Center • Off-Site Backup • Staff $$$$ $$$$ $$$$
  • 9.
    Data Warehouse Unit(DWU) Simply buy the query performance you need, not just hardware Quantified by workload objectives: how fast rows are scanned, loaded, copied Measure of Power Transparency First DW service to offer compute power on demand, independent of storageOn Demand Scan 1B rows 100 DWU = 297 sec 400 DWU = 74 sec 800 DWU = 37 sec 1,600 DWU = 19 sec Scan Rate xx M row/sec Loading Rate xx K row/sec Table Copy Rate xx K row/sec 100 DWU
  • 10.
  • 11.
    SQL Server Compatibility FamiliarSQL Server scripts and tools you have been using just work Modular programming (write once, execute multiple times) Faster code execution Encapsulated programming logic Easier maintenance of large tables Improves performance Enhanced scalability and availability Allows proper use and comparisons of characters in different languages Mature Column-Store technology for best- in-class DW query performance
  • 12.
    Complete, consistent, scalabledata platform Azure Data Warehouse