Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What’s new in
SQL Server 2017
James Serra
Big Data Evangelist
Microsoft
JamesSerra3@gmail.com
About Me
 Microsoft, Big Data Evangelist
 In IT for 30 years, worked on many BI and DW projects
 Worked as desktop/web/...
The power of SQL Server
Microsoft Tableau Oracle
$120
$480
$2,230
Self-service BI per userat massive scale
0 1
4
0 0
3
0
3...
SQL Server 2017
Meeting you where you are
It’s the same SQL Server database engine with many features and services
availab...
How we develop SQL
Cloud-first but not cloud-only
Use SQL Database to improve core SQL Server features and cadence
Many in...
A consistent experience from SQL Server on-premises to
Microsoft Azure IaaS and PaaS
On-premises, private cloud, and publi...
SQL Server 2017
New Features
Database Engine new features
Linux/Docker support
• RHEL, Ubuntu, SLES, and Docker
containers
Adaptive Query Processing
• ...
Database Engine new features
Graph
• Store relationships using nodes/edges
• Analyze interconnected data using
node/edge q...
Database Engine new features
Enhanced performance for natively complied T-SQL modules
• OPENJSON, FOR JSON, JSON
• CROSS A...
Database Engine new features
Native scoring with T-SQL PREDICT
Resumable online index rebuild
• Pause/resume online index ...
Integration Services new features
Integration Services Scale Out
• Distribute SSIS package execution more easily across mu...
Analysis Services new features
Object level security for tabular models
Get Data enhancements
• New data sources
• Modern ...
Reporting Services new features
Comments
• Comments are now available for reports, to add perspective and
collaborate with...
Machine Learning Services new features
Python support
• Python and R scripts are now supported
• revoscalepy - Pythonic eq...
SQL Server 2017
on Linux
Power of the SQL Server Database Engine on the
platform of your choice
Linux distributions: RedHat Enterprise
Linux (RHEL)...
Linux-native user experience
Supported platforms
Platform Supported version(s) Supported file system(s)
Red Hat Enterprise Linux 7.3 XFS or EXT4
SUSE L...
Cross-System Architecture
SQL Platform Abstraction Layer
(SQLPAL)
RDBMS AS IS RS
Windows Linux
Windows
Host Ext.
Linux Hos...
Tools and programmability
• Windows-based SQL Server tools -
like SSMS, SSDT, Profiler – work when
connected to SQL Server...
Client Connectivity
Language Platform More Details
C# Windows, Linux, macOS Microsoft ADO.NET for SQL Server
Java Windows,...
What’s available on Linux?
Operations Features
• Support for RHEL, Ubuntu, SLES, Docker
• Package-based installs
• Support...
What’s available on Linux?
Programming Features
• All major language driver compatibility
• In-Memory OLTP
• Columnstore i...
Features not currently supported on Linux
In-Memory OLTP
In-Memory Online Transaction Processing (OLTP)
In-Memory OLTP is the premier technology available in SQL Server and Azure
...
In-Memory OLTP enhancements (SQL Server 2017)
 sp_spaceused is now supported for memory-optimized tables.
 sp_rename is ...
High Availability
Availability Groups + Failover Clustering (Linux)
Always On:
Failover Cluster Instances
and Availability Groups work
toget...
Always On cross-platform capabilities
Mission critical availability on any platform
• Always On availability groups
for Li...
Guarantee commits on
synchronous secondary replicas
Use REQUIRED_COPIES_TO_COMMIT with CREATE
AVAILABILITY GROUP or ALTER ...
CLUSTER_TYPE
CLUSTER_TYPE Use with CREATE AVAILABILITY
GROUP. Identifies the type of server cluster manager
that manages a...
Automatic Tuning
Automatic Tuning
Automatic plan correction identifies problematic plans and fixes SQL plan
performance problems.
Adapt
Ver...
Automatic Plan Choice Detection
sys.dm_db_tuning_recommendations
Automatic Plan Correction
sys.dm_db_tuning_recommendations by enabling the AUTOMATIC_TUNING
database property
ALTER DATABA...
Adaptive Query
Processing (AQP)
Adaptive Query Processing
Three features to improve query performance
Enabled when the database is in SQL Server 2017 comp...
Query Processing and Cardinality Estimation
When estimates are accurate (enough), we make informed decisions around
order ...
Common reasons for incorrect cardinality estimates
Missing
statistics
Stale statistics
Inadequate
statistics sample
rate
B...
Cost of incorrect estimates
Slow query
response time due
to inefficient plans
Excessive resource
utilization (CPU,
Memory,...
Interleaved Execution
Pre 2017
2017+
100 rows guessed
for MSTVFs
MSTVF
identified
500k rows
assumed
Performance
issues if ...
Batch Mode Memory Grant Feedback
Problem: Queries can spill to
disk or take too much memory
based on poor cardinality
esti...
Batch Mode Adaptive Joins
Problem: If cardinality
estimates are skewed, we
may choose an inappropriate
join algorithm
Batc...
Support for
Graph data in
SQL Server 2017
A graph is collection of Nodes and Edges
Undirected Graph
Directed Graph
Weighted Graph
Property Graph
What is a Graph?
No...
Typical Scenarios for Graph Databases
Hierarchical or interconnected
data, entities with multiple
parents.
Analyze interco...
Introducing SQL Server Graph
A collection of node and edge
tables in the database
Language Extensions
DDL Extensions – cre...
DDL Extensions
• Create node and edge tables
• Properties associated with
nodes and edges
CREATE TABLE Product (ID INTEGER...
DML Extensions
Multi-hop navigation and join-free
pattern matching using the MATCH
predicate:
SELECT Prod.name as ProductN...
Transact-SQL
Extensions in
SQL Server 2017
T-SQL TRIM
TRIM ( [ characters FROM ] string )
Removes the space character (char(32)) or other specified characters from t...
T-SQL CONCAT_WS
CONCAT_WS ( separator, argument1, argument2 [, argumentN]…)
Concatenates a variable number of arguments wi...
T-SQL TRANSLATE
TRANSLATE ( inputString, characters, translations)
Returns the string provided as a first argument after s...
T-SQL STRING_AGG
STRING_AGG ( expression, separator ) [ <order_clause> ]
<order_clause> ::=
WITHIN GROUP ( ORDER BY <order...
T-SQL BULK INSERT / OPENROWSET(BULK…)
[ [ , ] FORMAT = 'CSV' ]
[ [ , ] FIELDQUOTE = 'quote_characters']
Additional options...
Columnstore
Data stored as columns
SQL Server performance features: Columnstore
Columnstore
A technology for storing, retrieving, and
...
ColumnStore Index Enhancements (SQL Server 2017)
 Clustered Columnstore Indexes now support LOB columns (nvarchar(max), v...
Resumable
Online Indexing
Resumable Online Indexing
With Resumable Online Index Rebuild you can resume a paused index rebuild
operation from where t...
Using Resumable Online Index Rebuild
Start a resumable online index rebuild
ALTER INDEX test_idx on test_table REBUILD WIT...
Upgrading
and migrating
to SQL Server
2017
Upgrade and migration tools
Data Migration Assistant (DMA)
• Upgrade from previous version of SQL Server (on-premises or S...
Upgrading to SQL Server 2017
In-place or side-by-side upgrade path from:
• SQL Server 2008
• SQL Server 2008 R2
• SQL Serv...
Legacy SQL Server instance
DMA: Assess and upgrade schema
1. Assess and identify issues
2. Fix issues
3. Upgrade database
...
Choosing a migration target
“What’s the best path for me?”
Migrating to SQL Server 2017 from other platforms
Identify apps
For migration
Use migration tools
and partners
Deploy to
p...
Migration Assistant
Database and application migration process
• Database connectivity
• User Login and Permission
• Perfo...
SQL Server Migration Assistant (SSMA)
Automates and simplifies all phases of database migration
Assess migration complexit...
Using SQL Server Migration Assistant (SSMA)
SSMA: Automates components of database migrations to SQL Server
DB2, Oracle, S...
Azure solution paths
Do not have to manage any VMs, OS or database software,
including upgrades, high availability, and ba...
Azure migration tools & services
Assess Migrate
Legacy SQL Server instance
DMA: Assess and migrate schema
1. Assess and identify issues
2. Fix issues
3. Convert and
deplo...
Oracle SQL
SQL DB
Azure Database Migration Service
Accelerating your journey to the cloud
Streamline database migration to...
S E A M L E S S C LO U D
I N T E G R AT I O N
Easy lift-and-shift migration
Azure SQL Database Managed
Instance private pr...
Editions,
features, and
capacity
SQL Server Editions
SQL Server edition Definition
Enterprise
The premium offering, SQL Server Enterprise edition delivers ...
Capacity Limits by Edition
Feature Enterprise Standard Web Express
Maximum compute capacity used by a
single instance - SQ...
SQL Server Features
Server components Description
SQL Server Database Engine
SQL Server Database Engine includes the Datab...
Features by Edition
Some SQL Server features (and sub-features) are available
only to certain editions:
• see https://docs...
Q & A ?
James Serra, Big Data Evangelist
Email me at: JamesSerra3@gmail.com
Follow me at: @JamesSerra
Link to me at: www.l...
Upcoming SlideShare
Loading in …5
×

What’s new in SQL Server 2017

6,739 views

Published on

Covers all the new features in SQL Server 2017, as well as details on upgrading and migrating to SQL Server 2017 or to Azure SQL Database.

Published in: Technology
  • Excellent briefing on core features of SQL Server 2017. Thank You.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • For online course coupons and discount visit http://www.todaycourses.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Nice summary.. thanks James !!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

What’s new in SQL Server 2017

  1. 1. What’s new in SQL Server 2017 James Serra Big Data Evangelist Microsoft JamesSerra3@gmail.com
  2. 2. About Me  Microsoft, Big Data Evangelist  In IT for 30 years, worked on many BI and DW projects  Worked as desktop/web/database developer, DBA, BI and DW architect and developer, MDM architect, PDW/APS developer  Been perm employee, contractor, consultant, business owner  Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data World conference  Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions  Blog at JamesSerra.com  Former SQL Server MVP  Author of book “Reporting with Microsoft SQL Server 2012”
  3. 3. The power of SQL Server Microsoft Tableau Oracle $120 $480 $2,230 Self-service BI per userat massive scale 0 1 4 0 0 3 0 34 29 22 15 5 22 16 6 43 20 69 18 49 74 3 0 10 20 30 40 50 60 70 80 1 2 3 4 5 6 7 NIST comprehensive vulnerability database, June 2016 SQL Server Oracle MySQL2 SAP HANA 1M Predictions per second Any platform Any data 1010 0101 0010 {} Any language T-SQL Java PHP Node.js C/C++ C#/VB.NET Python Ruby #1 TPC-H non-clustered results, August 2017 #1 in 30TB, 10TB, 1TB TPC-H Windows Server 2016
  4. 4. SQL Server 2017 Meeting you where you are It’s the same SQL Server database engine with many features and services available for all your applications—regardless of your operational ecosystem Linux Any data Any application Anywhere Choice of platform T-SQL Java C/C++ C#/VB.NET PHP Node.js Python Ruby 1010 0101 0010 { }
  5. 5. How we develop SQL Cloud-first but not cloud-only Use SQL Database to improve core SQL Server features and cadence Many interesting and compelling on-premises  cloud scenarios SQL Server and APS Azure SQL Virtual Machines Azure SQL Database Azure SQL Data Warehouse
  6. 6. A consistent experience from SQL Server on-premises to Microsoft Azure IaaS and PaaS On-premises, private cloud, and public cloud SQL Server local (Windows and Linux), VMs (Windows and Linux), containers, and SQL Database Common development, management, and identity tools including Active Directory, Visual Studio, Hyper-V, and System Center Scalability, availability, security, identity, backup and restore, and replication Many data sources Reporting, integration, processing, and analytics All supported in the hybrid cloud Consistency and integration
  7. 7. SQL Server 2017 New Features
  8. 8. Database Engine new features Linux/Docker support • RHEL, Ubuntu, SLES, and Docker containers Adaptive Query Processing • Faster queries just by upgrading • Interleaved execution • Batch-mode memory grant feedback • Batch-mode adaptive joins
  9. 9. Database Engine new features Graph • Store relationships using nodes/edges • Analyze interconnected data using node/edge query syntax SELECT r.name FROM Person AS p, likes AS l1, Person AS p2, likes AS l2, Restaurant AS r WHERE MATCH(p-(l1)->p2-(l2)->r) AND p.name = 'Chris' Automatic Tuning • Automatic plan correction – identify, and optionally fix, problematic query execution plans causing query performance problems • Automatic index management – make index recommendations (Azure SQL Database only)
  10. 10. Database Engine new features Enhanced performance for natively complied T-SQL modules • OPENJSON, FOR JSON, JSON • CROSS APPLY operations • Computed columns New string functions • TRIM, CONCAT_WS, TRANSLATE, and STRING_AGG with support for WITHIN GROUP (ORDER BY) BULK IMPORT now supports CSV format and Azure Blob Storage as file source
  11. 11. Database Engine new features Native scoring with T-SQL PREDICT Resumable online index rebuild • Pause/resume online index rebuilds Clusterless read-scale availability groups • Unlimited, geo-distributed, linear read scaling P S1 S2 S3 S4
  12. 12. Integration Services new features Integration Services Scale Out • Distribute SSIS package execution more easily across multiple workers, and manage executions and workers from a single master computer Integration Services on Linux • TRIM, CONCAT_WS, TRANSLATE, and STRING_AGG with support for WITHIN GROUP (ORDER BY) Connectivity improvements • Connect to the OData feeds of Microsoft Dynamics AX Online and Microsoft Dynamics CRM Online with the updated OData components
  13. 13. Analysis Services new features Object level security for tabular models Get Data enhancements • New data sources • Modern experience for tabular models Enhanced ragged hierarchy support • New Hide Members property to hide blank members in ragged hierarchies
  14. 14. Reporting Services new features Comments • Comments are now available for reports, to add perspective and collaborate with others. You can also include attachments with comments Broader DAX support • Report Builder and SQL Server Data Tools, you can create native DAX queries against supported tabular data models by dragging and dropping desired fields in the query designers. Standalone installer • SSRS is no longer distributed through SQL Server setup
  15. 15. Machine Learning Services new features Python support • Python and R scripts are now supported • revoscalepy - Pythonic equivalent of RevoScaleR – parallel algorithms for data processing with a rich API MicrosoftML • Package of machine learning algorithms and transforms (with Python bindings), as well as pre-trained models for image extraction or sentiment analysis
  16. 16. SQL Server 2017 on Linux
  17. 17. Power of the SQL Server Database Engine on the platform of your choice Linux distributions: RedHat Enterprise Linux (RHEL), Ubuntu, and SUSE Linux Enterprise Server (SLES) Docker: Windows & Linux containers Windows Server / Windows 10 Linux Linux/Windows container Windows
  18. 18. Linux-native user experience
  19. 19. Supported platforms Platform Supported version(s) Supported file system(s) Red Hat Enterprise Linux 7.3 XFS or EXT4 SUSE Linux Enterprise Server v12 SP2 EXT4 Ubuntu 16.04 EXT4 Docker Engine (on Windows, Mac, or Linux) 1.8+ N/A System requirements for SQL Server on Linux
  20. 20. Cross-System Architecture SQL Platform Abstraction Layer (SQLPAL) RDBMS AS IS RS Windows Linux Windows Host Ext. Linux Host Extension SQL Platform Abstraction Layer (SQLPAL) Host Extension mapping to OS system calls (IO, Memory, CPU scheduling) Win32-like APIsSQL OS API SQL OS v2 Everything else System Resource & Latency Sensitive Code Paths
  21. 21. Tools and programmability • Windows-based SQL Server tools - like SSMS, SSDT, Profiler – work when connected to SQL Server on Linux • All existing drivers and frameworks supported • Third-party tools continue to work • Native command-line tools – sqlcmd, bcp • Visual Studio Code mssql extension
  22. 22. Client Connectivity Language Platform More Details C# Windows, Linux, macOS Microsoft ADO.NET for SQL Server Java Windows, Linux, macOS Microsoft JDBC Driver for SQL Server PHP Windows, Linux, macOS PHP SQL Driver for SQL Server Node.js Windows, Linux, macOS Node.js Driver for SQL Server Python Windows, Linux, macOS Python SQL Driver Ruby Windows, Linux, macOS Ruby Driver for SQL Server C++ Windows, Linux, macOS Microsoft ODBC Driver for SQL Server SQL Server client drivers are available for many programming languages, including:
  23. 23. What’s available on Linux? Operations Features • Support for RHEL, Ubuntu, SLES, Docker • Package-based installs • Support for Open Shift, Docker Swarm • Failover Clustering via Pacemaker • Backup / Restore • SSMS on Windows connected to Linux • Command line tools: sqlcmd, bcp • Transparent Data Encryption • Backup Encryption • SCOM Management Pack • DMVs • Table partitioning • SQL Server Agent • Full-Text search • Integration Services • Active Directory (integrated) authentication • TLS for encrypted connections
  24. 24. What’s available on Linux? Programming Features • All major language driver compatibility • In-Memory OLTP • Columnstore indexes • Query Store • Compression • Always Encrypted • Row-Level Security, Data Masking • Auditing • Service Broker • CLR • JSON, XML • Third-party tools
  25. 25. Features not currently supported on Linux
  26. 26. In-Memory OLTP
  27. 27. In-Memory Online Transaction Processing (OLTP) In-Memory OLTP is the premier technology available in SQL Server and Azure SQL Database for optimizing performance of transaction processing, data ingestion, data load, and transient data scenarios Memory-optimized tables outperform traditional disk-based tables, leading to more responsive transactional applications Memory-optimized tables also improve throughput and reduce latency for transaction processing, and can help improve performance of transient data scenarios such as temp tables and ETL
  28. 28. In-Memory OLTP enhancements (SQL Server 2017)  sp_spaceused is now supported for memory-optimized tables.  sp_rename is now supported for memory-optimized tables and natively compiled T-SQL modules.  CASE statements are now supported for natively compiled T-SQL modules.  The limitation of 8 indexes on memory-optimized tables has been eliminated.  TOP (N) WITH TIES is now supported in natively compiled T-SQL modules.  ALTER TABLE against memory-optimized tables is now substantially faster in most cases.  Transaction log redo of memory-optimized tables is now done in parallel. This bolsters faster recovery times and significantly increases the sustained throughput of AlwaysOn availability group configuration.  Memory-optimized filegroup files can now be stored on Azure Storage. Backup/Restore of memory-optimized files on Azure Storage is supported.  Support for computed columns in memory-optimized tables, including indexes on computed columns.  Full support for JSON functions in natively compiled modules, and in check constraints.  CROSS APPLY operator in natively compiled modules.  Performance of btree (non-clustered) index rebuild for MEMORY_OPTIMIZED tables during database recovery has been significantly optimized. This improvement substantially reduces the database recovery time when non-clustered indexes are used.
  29. 29. High Availability
  30. 30. Availability Groups + Failover Clustering (Linux) Always On: Failover Cluster Instances and Availability Groups work together to ensure data is accessible despite failures Pacemaker Cluster Network Subnet Network Subnet Storage Node NodeNodeNodeNode SQL Server Instance SQL Server Instance SQL Server Instance Always On SQL Server Failover Cluster Instance Secondary Replica Secondary Replica Secondary Replica Primary Replica Always On Availability Group Instance Network Name Pacemaker Configuration Pacemaker Configuration Pacemaker Configuration Pacemaker Configuration Pacemaker Configuration Instance Network Name Instance Network Name Instance Network Name Pacemaker cluster virtual IP Storage Storage Shared Storage DNS name (manual registration)
  31. 31. Always On cross-platform capabilities Mission critical availability on any platform • Always On availability groups for LinuxNEW* and Windows for HA and DR • Flexibility for HA architecturesNEW* • Ultimate HA with OS-level redundancy and failover • Load balancing of readable secondaries • High Availability • Offload Backups • Scale BI Reporting • Enables Testing • Enables Migrations
  32. 32. Guarantee commits on synchronous secondary replicas Use REQUIRED_COPIES_TO_COMMIT with CREATE AVAILABILITY GROUP or ALTER AVAILABILITY GROUP. When REQUIRED_COPIES_TO_COMMIT is set to a value higher than 0, transactions at the primary replica databases will wait until the transaction is committed on the specified number of synchronous secondary replica database transaction logs. If enough synchronous secondary replicas are not online, write transactions to primary replica will stop until communication with sufficient secondary replicas resume. Enhanced AlwaysOn Availability Groups (SQL Server 2017) AG_Listener New York (Primary) Asynchronous data Movement Synchronous data Movement Unified HA solution Hong Kong (Secondary) New Jersey (Secondary)
  33. 33. CLUSTER_TYPE CLUSTER_TYPE Use with CREATE AVAILABILITY GROUP. Identifies the type of server cluster manager that manages an availability group. Can be one of the following types: WSFC - Windows server failover cluster. On Windows, it is the default value for CLUSTER_TYPE. EXTERNAL - A cluster manager that is not Windows server failover cluster - for example, on Linux with Pacemaker. NONE - No cluster manager. Used for a read- scale availability group. Enhanced AlwaysOn Availability Groups (SQL Server 2017) AG_Listener New York (Primary) Asynchronous data Movement Synchronous data Movement Unified HA solution Hong Kong (Secondary) New Jersey (Secondary)
  34. 34. Automatic Tuning
  35. 35. Automatic Tuning Automatic plan correction identifies problematic plans and fixes SQL plan performance problems. Adapt Verify Learn
  36. 36. Automatic Plan Choice Detection sys.dm_db_tuning_recommendations
  37. 37. Automatic Plan Correction sys.dm_db_tuning_recommendations by enabling the AUTOMATIC_TUNING database property ALTER DATABASE current SET AUTOMATIC_TUNING ( FORCE_LAST_GOOD_PLAN = ON );
  38. 38. Adaptive Query Processing (AQP)
  39. 39. Adaptive Query Processing Three features to improve query performance Enabled when the database is in SQL Server 2017 compatibility mode (140) ALTER DATABASE current SET COMPATIBILITY_LEVEL = 140; Adaptive Query Processing Interleaved Execution Batch Mode Memory Grant Feedback Batch Mode Adaptive Joins
  40. 40. Query Processing and Cardinality Estimation When estimates are accurate (enough), we make informed decisions around order of operations and physical algorithm selection CE uses a combination of statistical techniques and assumptions During optimization, the cardinality estimation (CE) process is responsible for estimating the number of rows processed at each step in an execution plan
  41. 41. Common reasons for incorrect cardinality estimates Missing statistics Stale statistics Inadequate statistics sample rate Bad parameter sniffing scenarios Out-of-model query constructs • E.g. MSTVFs, table variables, XQuery Assumptions not aligned with data being queried • E.g. independence vs. correlation
  42. 42. Cost of incorrect estimates Slow query response time due to inefficient plans Excessive resource utilization (CPU, Memory, IO) Spills to disk Reduced throughput and concurrency T-SQL refactoring to work around off- model statements
  43. 43. Interleaved Execution Pre 2017 2017+ 100 rows guessed for MSTVFs MSTVF identified 500k rows assumed Performance issues if skewed Execute MSTVF Good performance Problem: Multi-statement table valued functions (MSTVFs) are treated as a black box by QP and we use a fixed optimization guess Interleaved Execution will materialize row counts for multi-statement table valued functions (MSTVFs) Downstream operations will benefit from the corrected MSTVF cardinality estimate
  44. 44. Batch Mode Memory Grant Feedback Problem: Queries can spill to disk or take too much memory based on poor cardinality estimates Memory Grant Feedback (MGF) will adjust memory grants based on execution feedback MGF will remove spills and improve concurrency for repeating queries
  45. 45. Batch Mode Adaptive Joins Problem: If cardinality estimates are skewed, we may choose an inappropriate join algorithm Batch Mode Adaptive Joins (AJ) will defer the choice of hash join or nested loop until after the first join input has been scanned AJ uses nested loop for small inputs, hash joins for large inputs Build input Adaptive threshold Hash join Nested loop Yes No
  46. 46. Support for Graph data in SQL Server 2017
  47. 47. A graph is collection of Nodes and Edges Undirected Graph Directed Graph Weighted Graph Property Graph What is a Graph? Node Person Person
  48. 48. Typical Scenarios for Graph Databases Hierarchical or interconnected data, entities with multiple parents. Analyze interconnected data, materialize new information from existing facts. Identify non-obvious connections Complex many-to-many relationships. Organically grow connections as the business evolves. A
  49. 49. Introducing SQL Server Graph A collection of node and edge tables in the database Language Extensions DDL Extensions – create node and edge tables DML Extensions – SELECT - T-SQL MATCH clause to support pattern matching and traversals; DELETE, UPDATE, and INSERT support graph tables Graph support is integrated into the SQL Server ecosystem Database Contains Graph isCollectionOf Node table has Properties Edge table may or may not have Properties Node Table(s) Edges connect Nodes Edge Table(s)
  50. 50. DDL Extensions • Create node and edge tables • Properties associated with nodes and edges CREATE TABLE Product (ID INTEGER PRIMARY KEY, name VARCHAR(100)) AS NODE; CREATE TABLE Supplier (ID INTEGER PRIMARY KEY, name VARCHAR(100)) AS NODE; CREATE TABLE hasInventory AS EDGE; CREATE TABLE located_at(address varchar(100)) AS EDGE;
  51. 51. DML Extensions Multi-hop navigation and join-free pattern matching using the MATCH predicate: SELECT Prod.name as ProductName, Sup.name as SupplierName FROM Product Prod, Supplier Sup, hasInventory hasIn, located_at supp_loc, Customer Cus, located_at cust_loc, orders, location loc WHERE MATCH( cus-(orders)->Prod<-(hasIn)-Sup AND cus-(cust_loc)->location<-(supp_loc)-Sup ) ;
  52. 52. Transact-SQL Extensions in SQL Server 2017
  53. 53. T-SQL TRIM TRIM ( [ characters FROM ] string ) Removes the space character (char(32)) or other specified characters from the start or end of a string. SELECT TRIM (' test ') AS Result ; Result ----------- test Removing the space character from both sides of string (equivalent to LTRIM(RTRIM(string))) SELECT TRIM( '.,! ' FROM '# test .') AS Result; Result --------------- # test Removes specified characters from both sides of string (Trimming multiple characters)
  54. 54. T-SQL CONCAT_WS CONCAT_WS ( separator, argument1, argument2 [, argumentN]…) Concatenates a variable number of arguments with a delimiter specified in the first argument. SELECT CONCAT_WS( ' - ','one','two','three','four') AS Result ; Result ------------------------ one - two - three - four Concatenating with a delimiter SELECT CONCAT_WS( ' - ','one',NULL,'two',NULL,'three',NULL,'four') AS Result ; Result --------------------------------- one - two - three - four Concatenation ignores NULL
  55. 55. T-SQL TRANSLATE TRANSLATE ( inputString, characters, translations) Returns the string provided as a first argument after some characters specified in the second argument are translated into a destination set of characters. SELECT TRANSLATE('2*[3+4]/{7-2}', '[]{}', '()()') AS Result ; Result ------------- 2*(3+4)/(7-2) Replace square and curly braces with regular braces SELECT TRANSLATE('[137.4, 72.3]' , '[,]', '( )') AS Point, TRANSLATE('(137.4 72.3)' , '( )', '[,]') AS Coordinates ; Point Coordinates ------------- ------------- (137.4 72.3) [137.4,72.3] Convert GeoJSON points into WKT
  56. 56. T-SQL STRING_AGG STRING_AGG ( expression, separator ) [ <order_clause> ] <order_clause> ::= WITHIN GROUP ( ORDER BY <order_by_expression_list> [ ASC | DESC ] ) Concatenates the values of string expressions and places separator values between them. The separator is not added at the end of string. SELECT STRING_AGG ( ISNULL(FirstName,'N/A'), ',') AS csv FROM Person.Person ; csv ----------------------------------------------------------------------------------------------------------------------------- Syed,Catherine,Kim,Kim,Kim,Hazem,Sam,Humberto,Gustavo,Pilar,Pilar,Aaron,Adam,Alex,Alexandra,Allison,Amanda,Amber,Andrea,Angel Generate list of names separated with comma (without NULL values) SELECT town, STRING_AGG (email, ';') WITHIN GROUP (ORDER BY email ASC) AS emails FROM dbo.Employee GROUP BY town ; town emails ------- --------------------------------------------------------------------------------- Seattle catherine0@adventure-works.com;kim2@adventure-works.com;syed0@adventure-works.com LA hazem0@adventure-works.com;sam1@adventure-works.com Generate a sorted list of emails per towns
  57. 57. T-SQL BULK INSERT / OPENROWSET(BULK…) [ [ , ] FORMAT = 'CSV' ] [ [ , ] FIELDQUOTE = 'quote_characters'] Additional options added that provide support for CSV format data files Data files and format files can now be loaded from Azure Blob Storage
  58. 58. Columnstore
  59. 59. Data stored as columns SQL Server performance features: Columnstore Columnstore A technology for storing, retrieving, and managing data by using a columnar data format called a columnstore. You can use columnstore indexes for real-time analytics on your operational workload. Key benefits Provides a very high level of data compression, typically 10x, to reduce your data warehouse storage cost significantly. Indexing on a column with repeated values vastly improves performance for analytics. Improved performance: More data fits in memory Batch-mode execution
  60. 60. ColumnStore Index Enhancements (SQL Server 2017)  Clustered Columnstore Indexes now support LOB columns (nvarchar(max), varchar(max), varbinary(max)).  Online non-clustered columnstore index build and rebuild support added
  61. 61. Resumable Online Indexing
  62. 62. Resumable Online Indexing With Resumable Online Index Rebuild you can resume a paused index rebuild operation from where the rebuild operation was paused rather than having to restart the operation at the beginning. In addition, this feature rebuilds indexes using only a small amount of log space. • Resume an index rebuild operation after an index rebuild failure, such as after a database failover or after running out of disk space. There is no need to restart the operation from the beginning. This can save a significant amount of time when rebuilding indexes for large tables. • Pause an ongoing index rebuild operation and resume it later – for example, to temporarily free up system resources to execute a high priority. Instead of aborting the index rebuild process, you can pause the index rebuild operation and resume it later without losing prior progress. • Rebuild large indexes without using a lot of log space and have a long-running transaction that blocks other maintenance activities. This helps log truncation and avoid out-of-log errors that are possible for long-running index rebuild operations.
  63. 63. Using Resumable Online Index Rebuild Start a resumable online index rebuild ALTER INDEX test_idx on test_table REBUILD WITH (ONLINE=ON, RESUMABLE=ON) ; Pause a resumable online index rebuild ALTER INDEX test_idx on test_table PAUSE ; Resume a paused online index rebuild ALTER INDEX test_idx on test_table RESUME ; Abort a resumable online index rebuild (which is running or paused) ALTER INDEX test_idx on test_table ABORT ; View metadata about resumable online index operations SELECT * FROM sys.index_resumable_operations ;
  64. 64. Upgrading and migrating to SQL Server 2017
  65. 65. Upgrade and migration tools Data Migration Assistant (DMA) • Upgrade from previous version of SQL Server (on-premises or SQL Server 2017 in Azure VM) SQL Server Migration Assistant • Migrate from Oracle, MySQL, SAP ASE, DB2, or Access to SQL Server 2017 (on-premises or SQL Server 2017 in Azure VM) Azure Database Migration Service • Migrate from SQL Server, Oracle, or MySQL to Azure SQL Database or SQL Server 2017 in Azure VM
  66. 66. Upgrading to SQL Server 2017 In-place or side-by-side upgrade path from: • SQL Server 2008 • SQL Server 2008 R2 • SQL Server 2012 • SQL Server 2014 • SQL Server 2016 Side-by-side upgrade path from: • SQL Server 2005 Use Data Migration Assistant to prepare for migration
  67. 67. Legacy SQL Server instance DMA: Assess and upgrade schema 1. Assess and identify issues 2. Fix issues 3. Upgrade database Data Migration Assistant SQL Server 2017
  68. 68. Choosing a migration target “What’s the best path for me?”
  69. 69. Migrating to SQL Server 2017 from other platforms Identify apps For migration Use migration tools and partners Deploy to production SQL Server Migration Assistant Global partner ecosystem AND SQL Server 2017 on Windows SQL Server 2017 on Linux OR
  70. 70. Migration Assistant Database and application migration process • Database connectivity • User Login and Permission • Performance Tuning • Database Discovery • Architecture requirements • (HADR, performance, locale, maintenance, dependencies, etc.) • Migration Assessment • Complexity, Effort, Risk • Schema Conversion • Data Migration • Embedded SQL Statements • ETL and Batch • System and DB Interfaces
  71. 71. SQL Server Migration Assistant (SSMA) Automates and simplifies all phases of database migration Assess migration complexityMigration Analyzer Convert schema and business logicSchema Converter Migrate dataData Migrator Supports migration from DB2, Oracle, SAP ASE, MySQL, or Access to SQL Server Validate converted database codeMigration Tester
  72. 72. Using SQL Server Migration Assistant (SSMA) SSMA: Automates components of database migrations to SQL Server DB2, Oracle, Sybase, Access, and MySQL analyzers are available Assess the migration project Migrate schema and business logic Migrate data Convert the application Test, integrate, and deploy SSMA migration analyzer SSMA data migrator SSMA schema converter
  73. 73. Azure solution paths Do not have to manage any VMs, OS or database software, including upgrades, high availability, and backups. Highly customized system to address the application’s specific performance and availability requirements.
  74. 74. Azure migration tools & services Assess Migrate
  75. 75. Legacy SQL Server instance DMA: Assess and migrate schema 1. Assess and identify issues 2. Fix issues 3. Convert and deploy schema DMA
  76. 76. Oracle SQL SQL DB Azure Database Migration Service Accelerating your journey to the cloud Streamline database migration to Azure SQL Database (PaaS) Managed service platform for migrating databases Migrate SQL Server & 3rd party databases to Azure SQL Database
  77. 77. S E A M L E S S C LO U D I N T E G R AT I O N Easy lift-and-shift migration Azure SQL Database Managed Instance private preview facilitates lift and shift migration from on- premises SQL Server to cloud Azure Hybrid Benefit for SQL Server maximizes current on-premises license investments to facilitate migration Database Migration Service (DMS) private preview provides seamless and reliable migration at scale with minimal downtime Most consistent data platform Database Migration Ser vice (DMS) Azure SQL Database Managed Instance Azure Hybrid Benefit (AHB) for SQL Ser ver
  78. 78. Editions, features, and capacity
  79. 79. SQL Server Editions SQL Server edition Definition Enterprise The premium offering, SQL Server Enterprise edition delivers comprehensive high-end datacenter capabilities with blazing-fast performance, unlimited virtualization, and end-to-end business intelligence — enabling high service levels for mission-critical workloads and end user access to data insights. Standard SQL Server Standard edition delivers basic data management and business intelligence database for departments and small organizations to run their applications and supports common development tools for on-premise and cloud — enabling effective database management with minimal IT resources. Web SQL Server Web edition is a low total-cost-of-ownership option for Web hosters and Web VAPs to provide scalability, affordability, and manageability capabilities for small to large scale Web properties. Developer SQL Server Developer edition lets developers build any kind of application on top of SQL Server. It includes all the functionality of Enterprise edition, but is licensed for use as a development and test system, not as a production server. SQL Server Developer is an ideal choice for people who build SQL Server and test applications. Express Express edition is the entry-level, free database and is ideal for learning and building desktop and small server data- driven applications. It is the best choice for independent software vendors, developers, and hobbyists building client applications. If you need more advanced database features, SQL Server Express can be seamlessly upgraded to other higher end versions of SQL Server. SQL Server Express LocalDB, a lightweight version of Express that has all of its programmability features, yet runs in user mode and has a fast, zero-configuration installation and a short list of prerequisites.
  80. 80. Capacity Limits by Edition Feature Enterprise Standard Web Express Maximum compute capacity used by a single instance - SQL Server Database Engine Operating system maximum Limited to lesser of 4 sockets or 24 cores Limited to lesser of 4 sockets or 16 cores Limited to lesser of 1 socket or 4 cores Maximum compute capacity used by a single instance - Analysis Services or Reporting Services Operating system maximum Limited to lesser of 4 sockets or 24 cores Limited to lesser of 4 sockets or 16 cores Limited to lesser of 1 socket or 4 cores Maximum memory for buffer pool per instance of SQL Server Database Engine Operating System Maximum 128 GB 64 GB 1410 MB Maximum memory for Columnstore segment cache per instance of SQL Server Database Engine Unlimited memory 32 GB 16 GB 352 MB Maximum memory-optimized data size per database in SQL Server Database Engine Unlimited memory 32 GB 16 GB 352 MB Maximum relational database size 524 PB 524 PB 524 PB 10 GB
  81. 81. SQL Server Features Server components Description SQL Server Database Engine SQL Server Database Engine includes the Database Engine, the core service for storing, processing, and securing data, replication, full-text search, tools for managing relational and XML data, in database analytics integration, and Polybase integration for access to Hadoop and other heterogeneous data sources, and the Data Quality Services (DQS) server. Analysis Services Analysis Services includes the tools for creating and managing online analytical processing (OLAP) and data mining applications. Reporting Services Reporting Services includes server and client components for creating, managing, and deploying tabular, matrix, graphical, and free-form reports. Reporting Services is also an extensible platform that you can use to develop report applications. Integration Services Integration Services is a set of graphical tools and programmable objects for moving, copying, and transforming data. It also includes the Data Quality Services (DQS) component for Integration Services. Master Data Services Master Data Services (MDS) is the SQL Server solution for master data management. MDS can be configured to manage any domain (products, customers, accounts) and includes hierarchies, granular security, transactions, data versioning, and business rules, as well as an Add-in for Excel that can be used to manage data. Machine Learning Services (In-Database) Machine Learning Services (In-Database) supports distributed, scalable machine learning solutions using enterprise data sources. SQL Server 2017 supports R and Python. Machine Learning Server (Standalone) Machine Learning Server (Standalone) supports deployment of distributed, scalable machine learning solutions on multiple platforms and using multiple enterprise data sources, including Linux, Hadoop, and Teradata. SQL Server 2017 supports R and Python.
  82. 82. Features by Edition Some SQL Server features (and sub-features) are available only to certain editions: • see https://docs.microsoft.com/en-us/sql/sql-server/editions-and- components-of-sql-server-2017 for a complete list
  83. 83. Q & A ? James Serra, Big Data Evangelist Email me at: JamesSerra3@gmail.com Follow me at: @JamesSerra Link to me at: www.linkedin.com/in/JamesSerra Visit my blog at: JamesSerra.com (where this slide deck is posted under the “Presentations” tab)

×