This document provides an overview of Microsoft Azure's database offerings, including SQL Server, Azure SQL Database, SQL Server on Azure Virtual Machines, Azure SQL Data Warehouse, and the Analytics Platform System. It describes features and differences between products, and includes sections on SQL Data Warehouse architecture, pricing, and PolyBase technology.

1. Microsoft Azure
Database Offerings
An introduction

3. SQL Server
▪ SQL Server
▪ Azure SQL Database
▪ SQL Server on Azure Virtual
Machines
▪ Azure SQL Data Warehouse
▪ Analytics Platform System (on-
premises appliance)
Other SQL Server Family Products
▪ SSIS ~ Data Factory*
▪ SSRS ~ Power BI*
▪ SSAS – Azure Analysis Service
* Close match interms of feature, but they are
diff products.

4. SQL Server
▪ SQL Server
▪ SQL Server on Azure Virtual Machines - PAAS
▪ Azure SQL Database – SAAS (Limited feature)
▪ Azure SQL Data Warehouse – Cloud Data Management Service
▪ Analytics Platform System – Turn Key Big Data Solution

5. SQL Server
Data Warehouse
Fully managed, petabyte-scale cloud data warehouse.

6. SQL Datawarehouse

7. SQL Server Datawarehouse
▪ Massively parallel processing architecture
▪ Combines the SQL Server relational database with Azure cloud scale-out
capabilities.
▪ Decouples storage from compute.
▪ Enables increasing, decreasing, pausing, or resuming compute.
▪ Integrates across the Azure platform.
▪ Utilizes SQL Server Transact-SQL (T-SQL) and tools.

8. Architecture

9. SQL Server vs SQL Data Warehouse
▪ SQL Data Warehouse is measured in Data Warehouse Units (DWUs).
▪ WUs are a measure of underlying resources like CPU, memory, IOPS
▪ Stored procedures, user-defined functions, table partitioning, indexes, and
collations are supported
▪ Columnstore indexes, PolyBase integration, and Data auditing supported.
▪ Could be connected to Data Factory, Stream Analytics, Machine Learning, and
Power BI

10. Logical Overview

11. Distributed queries

12. SQL DW Pricing
▪ Compute resources are represented by DWUs. $1.513/hr for 100 DWU in West
US region
▪ Data storage is charged based on Azure Premium Storage rates of
$135.17/1TB/month ($0.19/1TB/hour). Data storage includes the size of your data
warehouse and 7-days of incremental snapshot storage.
▪ Pricing Calculator

13. SQL vs Data Warehouse

14. Price campare between SQL Azure vs SQL DW
▪ DWU vs DTU-Database Transaction Units
▪ DW pricing is exclusively based on DWU, Storage charged independently.
▪ In Azure SQL pricing DTU rate changes for diff tier (database size)
▪ The two systems are not same. SQL Azure is a OLTP whereas SQL DW is a
analytics workload. (MPP architecture)

15. Demo
▪ Data Lake
▪ AzCopy
▪ BCP
▪ Data Factory
▪ Polybase

16. Microsoft Analytics
Platform
Massively parallel processing integrated system

17. Microsoft Analytics Platform
▪ Turn key – Big data solution
▪ aka Parallel Data Warehouse (PDW) + HDInsights + Polybase

18. Polybase
technology that accesses and combines both non-relational and relational data

19. Create External Data Source
CREATE EXTERNAL DATA SOURCE datasource_name
{WITH (
TYPE = <data_source>,
LOCATION =‘<location>’,
[JOB_TRACKER_LOCATION = ‘<jb_location>’]
};
Type of external data source1
Location of external data source2
Enabling or disabling of
MapReduce job generation
3

20. Create External File Format
CREATE EXTERNAL FILE FORMAT fileformat_name
{WITH (
FORMAT_TYPE = <type>,
[SERDE_METHOD = ‘<sede_method>’,]
[DATA_COMPRESSION = ‘<compr_method>’,
[FORMAT_OPTIONS (<format_options>)]
};
Type of external data source1
(De)Serialization method [Hive
RCFile]
2
Compression method3
(Optional) Format Options [Text
Files]
4

21. <Format Options> :: =
[,FIELD_TERMINATOR = ‘value’],
[,STRING_DELIMITER = ‘value’],
[,DATE_FORMAT = ‘value’],
[USE_TYPE_DEFAULT = ‘value’]
Format Options
Column delimiter1
Delimiter for string data types2
To specify a particular date format3
How missing entries are handled4

22. Create External Table
CREATE EXTERNAL TABLE table_name
({<column_definition>}[,..n ])
{WITH (
DATA_SOURCE = <data_source>,
FILE_FORMAT = <file_format>,
LOCATION =‘<file_path>’,
[REJECT_VALUE = <value>],
…)};
Referencing external data source1
Referencing external file format2
Path of the Hadoop file/folder3
(Optional) Reject parameters4

23. Redis
High throughput and consistent low-latency data access Cache

24. Redis
▪ In-memory key-structure database
▪ Fire data strctures + publish/subscribe
▪ Strings, Lists, Sets, hashes, Sorted Sets (Zsets)
▪ Server-side scripting (like stored procedures) with Lua in Redis 2.6
▪ Persistence via snapshot or append-only file
▪ Replication supported
▪ High-availability and automated failover via 'Redis senti
http://try.redis.io/
Set Key Value
Get Key
Del
SETNX
INCR
Expire
RPUSH, LPUSH, LLEN,
LRANGE, LPOP, and
RPOP.
SADD, SREM,
SISMEMBER,
SMEMBERS and SUNION.

25. Document DB
Blazing fast, planet-scale NoSQL

26. Document DB
▪ DocumentDB is Microsoft’s NoSQL, JSON database for rapid development.
▪ Schema-less, NoSQL document database
▪ Fully managed, with provisioned capacity
▪ Stored entities are JSON documents
▪ Designed to scale into petabytes
▪ https://www.documentdb.com/sql/demo
• Database Account
• Database
• Collection
• Document
• Attachment
• Stored Procedure
• Trigger
• User-defined functions
• User
• Permission
• Media

27. Azure Data Factory
Create, schedule, and manage data pipelines

28. Data Factory
▪ Create, schedule, and manage data pipelines
▪ Visualize data lineage
▪ Connect to on-premises and cloud data sources
▪ Monitor data pipeline health
▪ Automate cloud resource management

29. Table Storage
NoSQL key-value store

30. Table Storage
▪ Store semi-structured data that’s highly available
▪ Create massively scalable apps
▪ Create apps that require a flexible data schema
▪ Use JSON to serialize data
▪ Perform OData-based queries
▪ Partition Key, Row Key, Data
▪ Microsoft Azure Storage Explorer