SlideShare a Scribd company logo
1 of 178
Download to read offline
Move a Successful SQL Server Transactional
Database to Azure
Ike Ellis, MVP
General Manager – Data & AI Practice
Solliance
@ike_ellis
www.ikeellis.com
youtube.com/IkeEllisOnTheMic
• Founder of San Diego
Power BI and PowerApps
UserGroup
• Founder of the San Diego
Software Architecture
Group
• MVP since 2011
• Author of Developing Azure
Solutions, Power BI MVP
Book
• Speaker at PASS Summit,
SQLBits, DevIntersections,
TechEd, Craft, Microsoft
Azure & AI Conference
Here’s what you get in this course
• All the slide decks
• Videos of every presentation that you can watch forever, share with friends, do what you want
with it
• Lab #1: Hybrid Azure/On-premise AlwaysOn Ags
• https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually-
configure-prerequisites-tutorial
• Lab #2: Microsoft Cloud Workshop
• https://microsoftcloudworkshop.com/
• Migrate SQL Server to Azure
• https://github.com/microsoft/MCW-Migrating-SQL-databases-to-Azure
• Migrate Oracle to Azure (Postgres or Azure SQL)
• https://github.com/Microsoft/MCW-Migrating-Oracle-to-Azure-SQL-and-PostgreSQL
Microsoft Migration Guides
• https://docs.microsoft.com/en-us/data-migration/
Agenda For The Course
• Introduction to Azure database options for SQL Server migrations
• Migration planning and introduction to migration techniques
• Migration preparation
• Migration to IAAS Azure VMs
• Migration to PAAS - Azure SQL Managed Instance
• Migration to PAAS - Azure SQL Database
• Post migration cleanup and migrating other things
• Platform modernization
• Call to action
Do you have something like this?
huge SQL
database
app 1
app 2
app 3
But really it’s like this:
huge SQL
database
app 1
app 2
app 3
Excel
SSRS
reports
SSIS etl
custom app
data dump to
vendor
SQL Agent job
that closes the
month
somebody hand made
a lookup table for one
report
linked server for
accounting process
stored procs
on a different
server
email
notification
for alerting
Power BI
Not a cloud app: reason #1: too expensive
• In the cloud, you pay for the following
• compute
• network
• disk
• memory
• But the most expensive thing by far:
• compute
• When the database is that large, you are paying for a lot
of compute for a database that is essentially hosting a lot
of data at rest
Not a cloud app: reason #2: too expensive
huge SQL
database
scales up as a unit. this is so big that if we need more power, we have to scale up
the entire huge thing
this tiny piece
here is really
important and
should scale
separately
Not a cloud app: reason #3: bad messaging
huge SQL
database
app 1
app 2
this is an expensive
and slow message
bus that takes a lot of
grooming.
you’re getting
overcharge for
storage and compute
Not a cloud app: reason #4: too hard to change
huge sql database
app 1
app 2
app 3
Excel
SSRS
reports
SSIS etl
custom app
data dump to
vendor
SQL Agent job
that closes the
month
somebody hand made
a lookup table for one
report
linked server for
accounting process
stored procs
on a different
server
email
notification
for alerting
Power BI
in order to
change it,
look at how
many things
have to be
tested and
verified and
deployed
together
not a cloud app: reason #4: too hard to deploy
huge SQL
database
app 1
app 2
app 3
Excel
SSRS
reports
SSIS etl
custom app
data dump to
vendor
SQL Agent job
that closes the
month
somebody hand made
a lookup table for one
report
linked server for
accounting process
stored procs on
a different
server
email
notification
for alerting
Power BI
in order to
deploy
this, look at
how many
things have
to be tested
and verified
and
deployed
together
in order to
upgrade this
to a new
version of
sql, look at
how many
things have
to be tested
and verified
and
upgraded
together
Not a cloud app: reason #4: too hard to upgrade
huge SQL
database
app 1
app 2
app 3
Excel
SSRS
reports
SSIS etl
custom app
data dump to
vendor
SQL Agent job
that closes the
month
somebody hand made
a lookup table for one
report
linked server for
accounting process
email
notification
for alerting
Power BI
Reason #5: not secure
huge SQL
database
pii data all up in
this
Reason #6: all kinds of performance
problems
huge sql database
For far too many years the answer to
what to do to solve all kinds of
performance problems was:
Throw hardware at it until it goes away
Now we have 32 cores with an average
cpu utilization of 2%
The cloud will punish you for decisions
like this
Simple example of a cloud-scale application
much smaller
SQL database
app 1
CosmosDB
app 2
much smaller
SQL database
app 3
How do they integrate?
much smaller
SQL database
app 1
CosmosDB
app 2
much smaller
SQL database
app 3
Azure SQL Options
• Azure SQL in a VM
• Azure SQL Database
• Single database
• Azure SQL Database Managed Instance
• Cross between SQL in a VM and Azure SQL
Database
• Azure SQL Elastic Pool
• Elastic pools are a collection of databases that share a set of resources,
providing an efficient, cost-effective solution for managing multiple databases
that have variable usage patterns
• Azure SQL Serverless
Azure SQL Database Options for Migration
Azure SQL
DB
A recommended migration path
On-premise
SQL Server
Azure VM
SQL Server
Planning for an On-Premise SQL Server
to Migrate to Azure
Ike Ellis, MVP
Solliance
General Manager – Data & AI Practice
Cloud Adoption Framework
• https://docs.microsoft.com/azure/cloud-adoption-framework/
• Rehost: This path involves the replacement of the old hosting environment with a new one, such as the Azure cloud, with minimal
changes to the architecture.
• Refactor: With this path, modifications are made to applications and databases to allow for the use of PaaS-based services.
• Rearchitect: This involves updating the parts of an application or database that are incompatible with moving to the cloud.
Frequently, this is due to past decisions made when initially building the applications and databases.
• Rebuild: This path involves rebuilding the app from scratch to support cloud-native concepts.
• Replace: If an app cannot move without significant time, effort, and expense, it may be time to replace the app with a new one that is
more cloud friendly such as a Software-as-a-Service (SaaS) based one.
Azure database migration journey
Assess Optimize
Migrate
Enables optimization
during or post migration
(fully managed service)
IaaS (virtual machines)
fall short here
Enables rehosting or
light refactoring for
most apps
Eliminates the need
to rearchitect or
rebuild your apps
SQ
L
Document and design your Azure management strategy
• Manage resources: Understand best practices for Azure resource groups and resources,
including smart naming, preventing accidental deletion, managing resource permissions, and
effective resource tagging.
• Use blueprints: Get a quick overview on using blueprints for building and managing your
deployment environments.
• Set up management groups: If you have multiple subscriptions, you can gather them
into management groups, and apply governance settings to those groups.
• Set up access policies: Apply compliance policies to your Azure resources.
• Design and, if practicable, test, a recovery plan: Put together a business continuity and disaster
recovery (BCDR) strategy to keep data safe, your environment resilient, and resources up and
running when outages occur.
• Manage VMs: Group VMs into availability groups for resilience and high availability. Use managed
disks for ease of VM disk and storage management.
• Monitor resource usage: Enable diagnostic logging for Azure resources, build alerts and playbooks
for proactive troubleshooting, and use the Azure dashboard for a unified view of your deployment
health and status.
• Manage support and updates: Understand your Azure support plan and how to implement it, get
best practices for keeping VMs up-to-date, and put processes in place for change management.
• Review architectures: Review sample Azure architectures to learn from as you build your post-
migration deployments.
Choose your cloud type
• Azure public Cloud
• Hybrid cloud
• Azure Government Cloud
Document your current on-premise architecture
Document how each component will go to the
cloud
Reasons to go to IAAS first – Look at what you get!
• No hardware to buy, manage, maintain and ultimately retire
• No management of the any supporting infrastructure
• Fast setup of environment and quick scaling
• Automatic upgrades, patching, and updates
• Integration with other services
• Built-in high-availability
• Automated backups
• Money-backed Service Level Agreements (SLA)
Things you will have to change when you go to IAAS
• Network configuration
• Installation of agents
• Different vCPU types
• Security systems
• Authentication methods
• SQL-based HADR
When to upgrade to SQL Server 2019
• Upgrade first
• Migrate to Azure
• Or Migrate to Azure
• Upgrade
But do not upgrade and migrate at the exact same time
Too much change. Too many variables. Too hard to rollback
What we learn when we migrate to IAAS first
• We’ve solved
• Networking
• Security
• Data migration
• We’ve learned
• Everything that was affected by doing that, including:
• What apps connected to it
• If those apps have the needed network latency
• Excel
• SSIS
• SSRS
• And we’ve catalogued and documented everything that we had to touch while doing the simple path first
Plan your licensing in the VM and Azure SQL DB MI
and Azure SQL DB
• Pay-as-you-go licensing allows businesses to use SQL Server images in
Azure on a pay-as-you-go basis, rather than requiring license keys.
• Bring-your-own-license (BYOL) is a model that allows organizations to take
advantage of SQL Server licenses they already own. Running SQL Server in
Azure can often provide even deeper discounts on licensing.
• Azure Hybrid Use Benefits may be available to companies with an existing
Enterprise Agreement (EA). To learn more, check out the Azure Hybrid
Benefits offering.
Reserve azure sql db resources in advance and
save up to 33%1
• budget and forecast better with upfront payment
for one-year or three-year terms
• get prioritized compute capacity in azure regions
• exchange or cancel reservations as your needs
evolve
• scale up or down within a performance tier and
region with auto-fit
• move saas apps between elastic pools and
single databases and keep your reserved
instance benefit
Reserved capacity for azure sql db
License included With Azure Hybrid Benefit
Up to 55%
savings2
Reserved Instances with
Azure Hybrid Benefit
Up to 80%
savings3
• Discounted rates up to 55% off to
support your ongoing development and
testing
• Dev/test pricing available for vcore-
based deployment options
• Eligible with active visual studio
subscription
• Your Visual Studio subscription comes
with $150/month of free Azure
Dev/test pricing for sql db
Most important! Hire your own Tim!
Prepare to Migrate an on-premise SQL
Server to Azure
Ike Ellis
Solliance
General Manager – Data + AI Practice
Microsoft MVP
First things first: let’s pay off some technical
debt
• before you can begin a cloud migration of a database, here are
some great things you can do to get ready for the project
Decouple from the server name
• Create a DNS CName (alias) for your sql server
• Change the ConnectionString for all applications, Excel, SSIS, SSRS, Power BI,
linked servers, backup tools, SSMS, SSAS, and everything else you can think of
• Make a list of every single location where you had to make the change
• This way when the server changes location, you can just change the cname
• If you really want to see if you succeeded at this or not, rename the server
• https://docs.microsoft.com/en-us/sql/database-engine/install-windows/rename-a-
computer-that-hosts-a-stand-alone-instance-of-sql-server?view=sql-server-ver15
App 1
DNS
SQL Server
DB
Connect
Decouple database names
• Look for three part object names in stored procedures in this
database and other databases
• worldwideimporters.dbo.customers
Make the database smaller
• Every time I ask developers or business users if we can delete some data, the
answer is always no
• When we actually look at the data, there is plenty we can either delete or move
• Really old data
• Audit logs and audit records
• Temp tables
• Tables that are empty or have really old data in them or rarely used data
• Reporting tables that really should be in a data warehouse
• Indexes that no one ever uses
• Remove overlapping indexes
Find unused indexes
SELECT
objects.name AS Table_name,
indexes.name AS Index_name,
dm_db_index_usage_stats.user_seeks,
dm_db_index_usage_stats.user_scans,
dm_db_index_usage_stats.user_updates
FROM
sys.dm_db_index_usage_stats
INNER JOIN sys.objects ON dm_db_index_usage_stats.OBJECT_ID = objects.OBJECT_ID
INNER JOIN sys.indexes ON indexes.index_id = dm_db_index_usage_stats.index_id AND dm_db_i
ndex_usage_stats.OBJECT_ID = indexes.OBJECT_ID
WHERE
AND
dm_db_index_usage_stats.user_lookups = 0
AND
dm_db_index_usage_stats.user_seeks = 0
AND
dm_db_index_usage_stats.user_scans = 0
ORDER BY
dm_db_index_usage_stats.user_updates DESC
Find overlapping indexes
select
s.Name + N'.' + t.name as [Table] ,i1.index_id as [Index1 ID], i1.name as [Index1 Name]
,dupIdx.index_id as [Index2 ID], dupIdx.name as [Index2 Name]
,c.name as [Column]
from
sys.tables t join sys.indexes i1 on
t.object_id = i1.object_id
join sys.index_columns ic1 on
ic1.object_id = i1.object_id and
ic1.index_id = i1.index_id and
ic1.index_column_id = 1
join sys.columns c on
c.object_id = ic1.object_id and
c.column_id = ic1.column_id
join sys.schemas s on
t.schema_id = s.schema_id
cross apply
(
select i2.index_id, i2.name
from
sys.indexes i2 join sys.index_columns ic2 on
ic2.object_id = i2.object_id and
ic2.index_id = i2.index_id and
ic2.index_column_id = 1
where
i2.object_id = i1.object_id and
i2.index_id > i1.index_id and
ic2.column_id = ic1.column_id
) dupIdx
order by s.name, t.name, i1.index_id
if the leading edge is the same,
the indexes can likely be
combined
Implement filtered indexes
My feeling is that every index should be filtered for legacy databases that have
been around for more than 3 years
create nonclustered index IDX_Data_Unprocessed_Filtered
on dbo.Data(RecId)
include(Processed)
where Processed = 0;
Make the database smaller: defrag and shrink
• Many of us have databases that have not been properly
maintained
• Defrag!
• Get rid of the free space in datafiles
• Find the worst offenders:
select f.type_desc as [Type]
,f.name as [FileName]
,fg.name as [FileGroup]
,f.physical_name as [Path]
,f.size / 128.0 as [CurrentSizeMB]
,f.size / 128.0 - convert(int,fileproperty(f.name,'SpaceUsed')) / 128.0 as [FreeSpaceMb]
from sys.database_files f with (nolock)
left outer join sys.filegroups fg with (nolock)
on f.data_space_id = fg.data_space_id option (recompile)
Make the database smaller: compression
• Introduce page compression
• Columnstore indexing
Make the database smaller: get the blobs out
• Blobs don’t belong in a database
• Move them to azure blob storage first!
• C# is probably how you got them in there, write a C# app to get them out and move
them to abs
• Save the URL to the file in the database as a varchar(100)
• Change the application to pull from abs instead of the database
• Benefits of getting the blobs
• Significantly reduces the size of the database
• Backed up automatically
• Much cheaper to save those files in abs than pay to have azure SQL database
store them
• Consistent load time no matter how many there are in there. SQL can’t say that
• With ABS we don’t pay for compute time! just storage!
• If you don’t do this, at least compress them where they are, but that’s a similar amount
of work
Find bad actors with the network
• Use profiler, extended events, wireshark, fiddler, chrome network tab
• Look for bad actors who are using too much network
• It’s easy to find pages that are doing the exact same code twice
• Or have n+1 problems
• Or are bringing back too much data and not paging it
• Saving the network will make your app more cloud-ready
Before the cloud migration, move your backup/restore process to
azure
• If you’re currently backing up your databases to an on-premise location, change that
process to move it to azure blob storage
• This is a great way to actually get your database to the cloud in BAK form
• Azure blob storage can be both a backup destination and a restore source!
Consider an on-premise upgrade of sql server
• Upgrades will get you on the latest version of the sql server optimizer and will
prepare you to use an azure sql database version of the optimizer
• Will give you features for compression and data management that you might
not have right now
Download some things
• Download and install the latest version of the MAP Toolkit.
• provides a readiness report to migrate the server
• Download and install the latest version of the Database Experimentation Assistant
• Download and install the Data Migration Assistant v3.3 or later
• Create an instance of sql server on Azure VMs by following the detail in the
article How to provision a Windows SQL Server virtual machine in the Azure portal
Azure Express Route
Take care of azure active directory
https://docs.microsoft.com/en-us/azure/active-directory/
Options for extending Active Directory
• AD Connect from a machine that’s on prem and that connects to azure active
directory
• Active Directory domain controller on a vm
• Create groups in ad on prem that map to what you want to secure in azure
• Add rights for resource groups
• Create simple groups on prem in ad that are the exact names of the resource
groups in your ad subscription
Why are we thinking about VMs?
• Even if you aren’t using IAAS, you will still need VMs
• Somewhere for SSMS/ADS
• Somewhere to move data around without getting charged ingress/egress
• Somewhere where the latency is native and close to the data center
• And you might go PAAS for the database, but go IAAS for SSRS, SSIS, SQL
Server Agent
• Networking can be a challenge
• Opening ports should be done sparingly
• A jump box on your VNet will let you manage Azure SQL DB MI
• You can pause it to stop getting billed for it
Disks
• SQL is I/O bound. I/O is incredibly important
It’s all about the disks (i/o) baby
Disk options: https://azure.microsoft.com/en-us/pricing/details/managed-disks/
• Premium SSD managed disks
• high performance, managed disks
• SSD
Ultra disks
not available in all regions
Storage space: ack!
• Stay away from storage spaces
• Taxes the cpu and that’s expensive
• Doesn’t really give you more IOPS
Diskspd https://github.com/microsoft/diskspd
azure VM: other disk recommendations
• Remember premium ssd or ultra ssd.
• Standard storage is only recommended for dev/test.
• Keep the storage account and vm in the same region.
• Disable azure geo-redundant storage (geo-replication) on
the storage account.
• Azure disk striping for performance
remember your disk best practices
• Use a minimum of 2 p30 disks (1 for log files and 1 for data files including tempdb).
• For workloads requiring ~50,000 IOPS, consider using an Ultra SSD.
• Avoid using operating system or temporary disks for database storage or logging.
• Enable read caching on the disk(s) hosting the data files and tempdb data files.
• Do not enable caching on disk(s) hosting the log file.
• important: Stop the SQL Server service when changing the cache settings for an
azure vm disk.
• Stripe multiple azure data disks to get increased IO throughput
• but be careful! you will load the cpu here!
• Format with documented allocation sizes.
• Place TempDB on the local ssd d: drive for mission critical sql server workloads (after
choosing correct VM size).
Some common things to watch out for in source
systems
• Single or dual disk SQL Server configuration where data and logs are mixed onto
single disk; you should explore breaking these apart.
• Single disk (operating system disk with data and log files) that is greater than
2TB
• Having a set of databases in which sum of data file size is greater than 32TB,
you will need to break them apart into separate disks for Azure Managed
Premium Disk support or you will be required to use the Ultra Disk option that
has limits up to 64TB.
• A single data or log file inside a managed disk that is greater than 4TB
• If you have more than 2PB of disk storage, you will need to break out your
database workloads onto multiple servers; or consider moving to a PaaS
• IOPS greater than 20,000 on single server instance; consider clustering
• Throughput higher than 900MB/sec
Choose the right VM size
• Watch the cpu/memory ratio
• Use Azure Migrate to determine hardware requirements
• Test them all!
• Not all types are available in all Azure regions, so choose your region
carefully
The importance of a performance baseline
Create a baseline, but also know what load your hardware can handle
Migrate an on-premise SQL Server to
IAAS – Azure VMs
Ike Ellis
Solliance
General Manager – Data + AI Practice
Microsoft MVP
Agenda
• Demo: Configure the VM
• Configure TempDB
• Demo backup and restore
• This is easier if you are already backing up and restoring to an Azure Blob Storage URL
• Demo: Backup/restore to URLs
• Demo: Migrating SQL Agent Jobs
• Demo: Migrating Logins
• Hybrid-cloud Azure AGs
Availability Zone
• Minimum of three separate zones in all enabled regions
• Each zone has it’s own power, cooling, and networking
• Zone-redundant services replicate your applications and data across Availability Zones to protect from single-points-of-failure
• If you create three or more VMs across three zones in an Azure region, your VMs are effectively distributed across three fault
domains and three update domains
Availability Set
• Ensures that each VM is in it’s own rack and it’s own update domain
• https://docs.microsoft.com/en-us/azure/virtual-machines/manage-availability
Configure TempDB
• Use Mike Petri’s script
• https://gist.github.com/FlogDonkey/3b9851f1c13b94f3f2d26b4ab5b49de0
Demo: Backup/Restore
Demo: Backup/Restore to Azure Blob Storage
Demo: Migrating SQL Agent Jobs
Demo: Migrating logins
• https://support.microsoft.com/en-us/help/918992/how-to-transfer-logins-and-passwords-between-instances-of-sql-server
No downtime migration: Azure AlwaysOn AGs
On-premise primary + Azure secondaries
• https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually-
configure-prerequisites-tutorial
• https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually-
configure-tutorial
Who is Azure SQL Database Managed Instance for?
customers looking to migrate a large number of
apps from on-premise or iaas, self-built or isv
provided, with as low migration effort as possible
& cost being a crucial factor
• Enable full isolation from other tenants without
resource sharing
• Promote secure communication over private IP
addresses with native vnet integration
• Enable your on-premise identities on cloud
instances, through integration with azure active
directory and ad connect
• Combine the best of SQL Server with the
benefits of a fully-managed service
• Use familiar SQL server features in SQL
database managed instance
What is the advantage of azure SQL DB MI over a
VM?
VNET support in SQL Database Managed Instance
Migration to PAAS
Azure SQL Database Managed
Instance
Ike Ellis
Solliance
General Manager – Data + AI Practice
Microsoft MVP
Put your DBs on autopilot and focus on your
business…
* - features coming soon
mi has it built-in
compute & storage provisioned on demand
fast & online scaling
full stack updates and patches
backups with health checks
point-in-time restore (configurable retention *)
99.99% availability with automatic failover
disaster recovery with single geo secondary (multiple*)
It’s much easier with the mi
compliance with all major industry standards
threat detection with automatic alerting
intelligent query processing
automatic performance tuning*
monitoring at scale with intelligent insights
data discovery and classification*
vulnerability assessment
Put your DBs on autopilot and focus on your
business…
* - features coming soon
• Fully-fledged SQL
instance with
nearly 100%
compat with on-
premise
• Built on the same
infrastructure as
SQL Database
• Provides the same
benefits (PaaS)
• Contained within
your VNet
• Private IP
addresses
• Express Route /
VPN connectivity
SQL Database
(PaaS)
Elastic
Pool
Managed
Instance
Single
database
• Transparent
• Frictionless
• Competitive
cross-db queries & transactions, linked servers to sql
.net clr modules
service broker
change data capture
transactional replication
Easily migrate from SQL Server & modernize
* - features coming soon
choice of instance collations* and instance time zone*
R services*
msdtc for distributed transactions
filestream / filetable, polybase
dmvs, xevents, query store
SQL agent and DB mail
sysadmin privileges and resource governor
Easily migrate from SQL Server & modernize
SQL auditing, rls
TDE, always encrypted, and dynamic data masking
built-in ha replaces on-prem setups
replace MDW with OMS monitoring
network security with vnets and private IPs
integrated auth. with azure ad
Isolation and connectivity of MI
mi: service tiers
Capability  Service tier General Purpose (GA) Business Critical (Public Preview)
Best for Apps with typical availability and
common IO latency requirements
Apps with highest availability and
lowest IO latency requirements.
Compute (vCores) 8, 16, 24, 32, 40, 64, 80 8, 16, 24, 32, 40, 64, 80
HA / Recovery Time Objective Remote storage based / Good Always On AG based / Better
Storage type / size Fast remote (Azure Premium) /
Up to 8 TB
Super-fast local SSD /
Up to 4 TB
Read scale out (read-only replica) No Yes
In-Memory OLTP No Yes
Price competitive with AWS? Yes, ~33% lower (license included) Yes, ~46% lower (license included)
General Purpose
Feature Description
Number of vCores* 8, 16, 24 (Gen 4)
8, 16, 24, 32, 40, 64, 80 (Gen 5)
SQL Server version / build SQL Server (latest available)
Min storage size 32 GB
Max storage size 8 TB
Max storage per database Determined by the max storage
size per instance
Expected storage IOPS 500-7500 IOPS per data file
(depends on data file).
See Premium Storage
Number of data files (ROWS)
per the database
Multiple
Number of log files (LOG)
per database
1
Managed automated backups Yes
HA Based on remote storage and
Azure Service Fabric
Built-in instance and database
monitoring and metrics
Yes
Automatic software patching Yes
VNet - Azure Resource
Manager deployment
Yes
VNet - Classic deployment
model
No
Portal support Yes
Business Critical
Feature Description
Number of vCores* 8, 16, 24, 32 (Gen 4)
8, 16, 24, 32, 40, 64, 80 (Gen 5)
SQL Server version / build SQL Server (latest available)
Additional features In-Memory OLTP
1 additional read-only replica (Read
Scale-Out)
Min storage size 32 GB
Max storage size •Gen 4: 1 TB (all vCore sizes
Gen 5:1 TB for 8, 16 vCores
•2 TB for 24 vCores
•4 TB for 32, 40, 64, 80 vCores
Max storage per database Determined by the max storage
size per instance
Number of data files (ROWS)
per the database
Multiple
Number of log files (LOG) per
database
1
Managed automated backups Yes
HA Based on Always On Availability
Groups and Azure Service Fabric
Built-in instance and database
monitoring and metrics
Yes
Automatic software patching Yes
VNet - Azure Resource
Manager deployment
Yes
VNet - Classic deployment
model
No
Portal support Yes
Business Critical service tier: collocated compute and storage
Primary endpoint
(read-write)
Read-only endpoint
Always On AG
SQL
SQL
SQL
SQL
Super-fast SSD
Primary replica Secondary replica
Secondary replica
Secondary replica
SQL DB deployment model overview
Azure SQL Database
Unit of
Monetization
Tiering
Pricing vs.
Competitors
 Basic: designed for apps with light workloads
 Standard: mid-level performance and business continuity
 Premium: low IO latency workloads and higher business continuity
 General Purpose
 “Business Critical”
Hybrid
Benefits
DTU – “Database Throughput Unit” –
measure of database performance
that blends CPU, memory and I/O.
 vCore for compute
 GBs for storage
 IOPs for IO
Yes, EE customers also get 4 cores in
General Purpose SKU
 Basic – very cheap because it priced to accommodate web customers
 Standard – comparable pricing but not easily explainable to customer
 Premium – expensive due to additional replicas and IOs
 Priced lower compared to AWS
 No
Best for
New apps, with a ‘one database per
app pattern’ and resources
guaranteed at DB level
Modernizing large number of existing
SQL Server apps from on-premises or
IaaS
New SaaS apps or modernizing existing
apps to SaaS, resource sharing across DBs
of existing LOB apps for higher efficiency
eDTU – elastic “Database Throughput
Unit” – measure of database
performance that blends CPU,
memory and I/O.
Deployment resources
portal deployment walk-through
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-create-tutorial-portal
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-vnet-configuration
arm template deployment walk-through
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/07/02/deploy-azure-sql-managed-instance-network-
environment-using-arm/
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/05/15/creating-azure-sql-managed-instance-using-
arm-templates/
Be empty:
Have specific route table:
Optional custom DNS:
No service endpoint:
Sufficient IP addresses:
Virtual network considerations
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-vnet-configuration#requirements
An MI instance must be deployed in an Azure virtual network
• allows for connecting directly from an on-premises network
• allows for connecting linked servers or other on-premises data stores
• allows for connecting to additional azure resources
Plan your deployment
• managed instance requires a minimum of 16 IP addresses in a subnet and may use up to 256 IP
addresses
• if deploying multiple managed instances inside the subnet, you need to optimize the subnet size
• the default values create a subnet that takes all the vnet address space, allowing for only mi inside the
virtual network
Routes
• Effective routes on the Managed Instance subnet are not supported
• Routes can be user-defined (UDR) or Border Gateway Protocol (BGP) routes propagated to network
interfaces through ExpressRoute or site-to-site VPN connections
• For BGP routes, create a 0.0.0.0/0 Next Hop Internet route and apply it to the Managed Instance
subnet
Network security groups (nsg)
NSGs on the Managed Instance subnet are not supported
Virtual network guidance
Considerations when creating a new virtual network for mi
• Calculate the subnet size
• Assess the needs for the rest of the vnet
• Disable service end points
• Create new arm virtual network
Virtual network guidance
Name Any valid name
Address space Any valid address range, such as 10.14.0.0/24
Subscription Your subscription
Resource Group Any valid resource group (new or existing)
Location Any valid location
Subnet name Any valid subnet name, such as mi_subnet
Subnet address range Any valid subnet address, such as 10.14.0.0/28. Use a subnet address space smaller than the address space itself to allow space
to create other subnets in the same VNet, such as a subnet for hosting test / client apps or gateway subnets to connect from on-
prem or other VNets.
Service endpoints Disabled
Built-in high availability
P
S S
Write
Write Ack
Ack
Read
write
Ack
value
DB
Availability group
Surface area of mi
•always on the latest and greatest SQL
engine version
•your code can be SQL deployment
model aware if necessary
MI is always on latest and greatest SQL engine version
documentation page
Your code can be SQL deployment model aware if
necessary
DB compatibility
based certification
•Microsoft database compatibility level
protection
•Easy to use tools to help you access
migration
Microsoft Database Compatibility Level Protection
Overall process
Contact Microsoft
App
compatibility:
what’s is
missing?
•Features with a better alternative in Azure
OMS
•Retired features
•Features considered post-GA
CLR considerations
Managed Instance cannot access file shares and Windows folders
Only CREATE ASSEMBLY FROM BINARY is supported
CREATE ASSEMBLY FROM FILE is not supported
ALTER ASSEMBLY can’t reference files
SQL Server Agent
Built into Managed Instance
Azure SQL Database requires using on-premises SQL Server Agent, Azure Automation, Elastic Jobs, or PowerShell
Always running
Services cannot be stopped or restarted like they can with on-premises
Option to auto-restart SQL Server if it stops unexpectedly is disabled
Option to auto-restart SQL Server Agent if it stops unexpectedly is disabled
Forwarding SQL Server events is disabled
On-premises SQL Server Agent allows for forwarding events to another server but this is currently not an option for a Managed Instance
Connection
Alias local host server is predefined for a Managed Instance, whereas on-premises SQL Server Agent allows that to be configured if needed
Creating jobs
Creating jobs is as simple and easy as on-premises
Jobs can be created using the UI or T-SQL
Alert System
Functions the same as on-premises for sending email alerts
SQLCMD
Cannot be called within a SQL Server Agent job
Can be used to connect to a Managed Instance
Service Broker within instances
Service broker is on by default for all user databases
Cross-instance service broker is not supported
CREATE ROUTE does not work with ADDRESS other than LOCAL
ALTER ROUTE does not work with ADDRESS other than LOCAL
Fully supported in Managed Instance
Functions the same as on-premises to set up and use
Azure SQL Database does not have Database Mail support
Database mail
Replication support
Supported
Snapshot replication. Same functionality as on-premises
Transactional replication
Unsupported
Peer-to-peer replication
Merge replication
Heterogeneous replication
Oracle publisher
For comparison, Azure SQL Database only supports being a transactional replication push subscriber
Some restrictions when used with a Managed Instance
Updatable subscriptions are not permitted
Publisher and distributor must be in the same location
If publisher and distributor are in a Managed Instance, Azure file share must be used to store data and schema from the publication
Connections to the Distributor must use SQL authentication
Additions to support Managed Instance
New fields have been added in replication-related tables in msdb
job_login, job_password, storage_connection_string
SSMS replication wizard supports using a Managed Instance
User DB file layout
considerations
Data file default initial size is 16MB with 16MB auto growth
These can and should be adjusted for your workload
File size limit is 8TB in General Purpose
Log file default initial size is 8MB with 16MB auto growth
This can and should be adjusted for your workload
Additional data files/filegroups can be added
Only using an ALTER DATABASE statement and the FILENAME clause is not permitted
Paths and File Names are chosen for you
Different from Azure SQL Database where additional files are not allowed
Multiple log files are not supported (and should not be needed)
A backup with multiple files/filegroups can be restored
Each user database has a FILESTREAM filegroup for In-Memory OLTP checkpoint files
Multiple log files are not supported (and should not be needed)
TempDB data file
considerations
Tempdb Tuning Options
Additional tempdb data files can be created if needed
Well-known tempdb tuning ‘fixes’ are on by default
Tempdb Resizing
Backups are
automatic
Database backup schedule is the same as Azure SQL
Database
COPY_ONLY, URL-based backups can be used to
perform manual full database backups
Backup retention is 7 days by default
Restore
considerations
Point-in-time restores are possible and must be performed manually
using the Azure Portal
Restoring automated backups from within SSMS is not allowed
You can only restore using the Azure Portal
COPY_ONLY, URL-based full backups can be restored using SSMS to
a Managed Instance only
Cannot be restored to on-premises as Managed Instance uses a
higher build than on-premises instances
Databases with multiple log files cannot be restored
Secondary log files must be removed prior to backing up and
restoring to a Managed Instance
Can restore backups in a specific DB Compatibility
Supports up to SQL 2005
Migrate an on-premise SQL Server to
Azure SQL Database
Ike Ellis
Solliance
General Manager – Data + AI Practice
Microsoft MVP
Agenda
• How Azure SQL DB Works
• Deployment models
• Purchasing options
• Service tiers
• Serverless compute
• Hyperscale
• T-SQL differences
• High availability
• Geo-replication
• DEMO: Configuring geo-replication
• Backup and restore
• Retention
• Accelerated Database Recovery
• Security
• BacPacs
• Demo: Scripted migration using bacpacs
How Azure SQL DB works
• logical vs physical server
Deployment models
• Single database
• Each database is isolated from others and is portable
• Elastic pool
Purchasing options
• vCore
• you choose the exact amount of compute resources that are always provisioned for your workload
• DTU
• bundled compute and storage packages balanced for common workloads
• usually cheaper
• only available with Azure SQL DB (not MI)
Service tiers
• General Purpose
• Business Critical
• Super-fast local SSDs
• Better log throughput
• 1 read-only replica built-in (can have 4 using geo-replication, but so can GP)
Serverless
• Automatically scales compute
• Minimum & maximum vCores
• Autopause delay
• Good for:
• Single databases with intermittent, unpredictable usage patterns interspersed with periods of inactivity
• New single databases without usage history where compute sizing is difficult or not possible to estimate prior to deployment in SQL Database.
Hyperscale
• Great solution for data marts and analytics systems –
good for loading because of low log contention
• Great solution for high-load, large transactional systems
T-SQL differences
• https://docs.microsoft.com/en-us/azure/azure-sql/database/transact-sql-tsql-differences-sql-server
High availability – General Purpose
High availability – Business Critical
Geo-replication
Backup/restore
• full backups every week
• differential backups every 12-24 hours
• transaction log backups every 5 to 10 minutes
• PITR
• Azure SQL Database retain sufficient backups to allow PITR within the last 7 days by default.
• You can change backup retention period per each active database in the 1-35 day range.
• You can configure full backup long-term retention (LTR) for up to 10 years in Azure Blob storage
Demo: Backup/restore and retention
Accelerated Database Recovery in Azure SQL
• Fast and consistent database recovery
• long running transactions do not impact the overall recovery time
• enabling fast and consistent database recovery irrespective of the number of active transactions in the system or their sizes.
• Instantaneous transaction rollback
• transaction rollback is instantaneous
• Aggressive log truncation
sLog
• a secondary in-memory log stream that stores log records for non-versioned operations (such as metadata cache invalidation, lock
acquisitions, and so on). The sLog is:
• Low volume and in-memory
• Persisted on disk by being serialized during the checkpoint process
• Periodically truncated as transactions commit
• Accelerates redo and undo by processing only the non-versioned operations
• Enables aggressive transaction log truncation by preserving only the required log records
•
Security
• Threat detection
• TLS
• TDE
• Azure Key Vault
• Always Encrypted
• Dynamic Data Masking
• Data discovery and classification
Migrating to Azure SQL Database using bacpac
• In practice, often includes multiple databases
• Scripts are far easier to implement when there are a lot of databases
Database compatibility considerations
• Feature compatibility
• https://docs.microsoft.com/en-us/azure/azure-sql/database/transact-sql-tsql-differences-sql-server#features-not-supported-in-sql-database
• Behavior compatibility
• While built on a common codebase, Azure SQL Database behaves differently from SQL Server in several important areas, i.e. high availability, backup,
connectivity, security, resource governance, default concurrency model, etc. Most importantly, the application must implement robust retry logic to
work reliably once migrated to the cloud
Test the migration, test the applications in Azure SQL
DB
• Use the DMA Assessment
• Build the project in SSDT
• Uses the same DacFX framework and bacpac file as Azure SQL DB
Migration size limits
• Diskspace requirements holding the bacpac file
• File has a hard time above 400GB
• Maximum size of a block blob in ABS is 4TB
Migration step detailed steps (most we’ve already
covered)
1. Provision Azure resources
2. Declare the start of application downtime.
3. Export a bacpac of the source database. This bacpac contains all database scoped objects and data to be migrated.
4. Upload the bacpac to Azure Blob Storage
5. Start and monitor a bacpac import operation.
6. Once the import operation completes successfully, verify objects and data in the migrated database.
7. Grant database access.
8. Verify application functionality, as well as database management and monitoring functionality.
9. Declare the end of application downtime.
Export the bacpac
• SqlPackage /Action:Export /SourceServerName:SampleSQLServer.sample.net,1433 /SourceDatabaseName:SampleDatabase
/TargetFile:"F:TempSampleDatabase.bacpac“
• All write access to the database should be stopped to keep the bacpac transactionally consistent
Upload the bacpac
• Needs a storage account
• AzCopy is a good, scriptable method for doing this
• AzCopy copy "D:Temptsqlv5.bacpac" "https://sqlbackupike.blob.core.windows.net/backups/?sv=2019-12-
12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2020-10-30T10:04:03Z&st=2020-09-28T02:04:03Z&sip=0.0.0.0-
255.255.255.255&spr=https&sig=iZW%2F6U7lxLvZO2MoRA8tdBshU2HOUwC2Fihyzhg%3D"
Import the bacpac
• Can do this in the portal
• Or script it:
• sqlpackage.exe /a:import /tcs:"Data Source=<serverName>.database.windows.net;Initial Catalog=<migratedDatabase>;User
Id=<userId>;Password=<password>" /sf:AdventureWorks2008R2.bacpac /p:DatabaseEdition=Premium
/p:DatabaseServiceObjective=P6
Verify migrated data
SELECT s.name AS [schema_name],
o.name AS [object_name],
o.type_desc AS [object_type_desc]
FROM sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.schema_id = s.schema_id
WHERE s.name <> 'sys'
AND
o.is_ms_shipped = 0
AND
o.type NOT IN ('IT','S')
;
Verify with row count
SELECT s.name AS [schema_name],
t.name AS table_name,
SUM(p.rows) AS row_count
FROM sys.partitions AS p
INNER JOIN sys.indexes AS i
ON p.object_id = i.object_id
AND
p.index_id = i.index_id
INNER JOIN sys.tables AS t
ON p.object_id = t.object_id
INNER JOIN sys.schemas AS s
ON t.schema_id = s.schema_id
WHERE s.name <> 'sys'
AND
t.is_ms_shipped = 0
AND
t.type NOT IN ('IT','S')
AND
i.type_desc IN ('HEAP','CLUSTERED','CLUSTERED COLUMNSTORE')
GROUP BY s.name, t.name;
Post migration tasks after migrating
databases to Azure offerings
Ike Ellis
Solliance
General Manager – Data + AI Practic
Microsoft MVP
You did it! You migrated! Now what?
Governance
• Governance is commonly defined as a set of policies, roles, responsibilities and processes that control a business system or systems
and how the various teams work together to achieve set goals
Governance: Security breaches
• How will you handle them?
• What is the response?
• Who is responsible?
• https://docs.microsoft.com/azure/governance/azure-management
Governance
• Will span the lifecycle of your Azure resources through the following major stages:
• Migrate : On-premises workload migrations
• Secure : Security and threat management
• Protect : Backup and recovery
• Monitor : Apps, infrastructure, network, logs and diagnostics
• Configure : Update management, automation and scripting
• Govern : Policy and cost management
Management
• Many management tasks can be performed, and automated, using the Azure VM Portal
• Register the SQL Server virtual machine in Azure with the SQL VM resource provider
• Creates the SQL Virtual Machine resource within the Azure subscription
• Registering the SQL Server on an Azure VM as a SQL Server VM resource may help streamline licensing and reduce costs.
SQL Server IAAS Agent Extension
• The SQL Server and Azure team developed a helpful provider and agent to support common administrative tasks
• The SQL Server IaaS Agent Extension runs on Azure virtual machines to automate administration tasks.
• Common tasks include:
• Automated backup
• Automated patching
• Azure Key Vault integration
Cost Optimization
• Azure Advisor
• a must-use tool for your Azure Subscriptions. It can help identify workloads that can be scaled down
• Azure Cost Management and Billing
• Tracks spending in your Azure subscriptions and resource groups
• Azure Budgets
• Proactively manage and monitor costs and get alerts on excessive spending
Rebuild all your indexes
• You may have new disks, new optimizer, new hardware
• Or a new product, like Azure SQL Database
Statistics
• Rebuild these too
• Particularly on stats that aren’t associated with indexing
BCDR Plan
• Business Continuity and Disaster Recovery
• Your plan has now changed that you are in Azure
• You need an updated plan that accounts for a new RTO and RPO
• Test the HADR plan
• On VMs, do you need to build an AG?
• If so, are you using Availability Sets or Availability Zones?
Backup/Restore strategy
• You need to document how you backup and restore
• When you restore
• What’s your Long Term Retention (LTR) policy
• How you are staying in compliance with it
• How you are testing it
• Test it regularly
Extra training
• If you want to gain valuable experience working with tools like Azure Site Recovery and Azure Backup, reference the Microsoft Cloud
Workshops and the Business Continuity and Disaster Recovery workshop.
Security - Azure Security Center
• Vulnerability assessment is an easy to configure service that can discover, track, and help you remediate potential database
vulnerabilities
• Advanced Threat Protection detects anomalous activities indicating unusual and potentially harmful attempts to access or exploit your
SQL server. - $15/month
• Advanced Data Security for SQL Servers
Virtual Machine - Antimalware
• You should install antimalware on all your virtual machines.
• Azure Security Center can also be used to deploy and integrated security solutions automatically to your Virtual Machine workloads
• Identify VMs that have public endpoints
Security: Just in time
• Remote Desktop is one of the biggest entry points for attackers.
• Brute force tools can continually attempt to login to the servers until a successful password has been found.
• Just in time (JIT) access can be enabled by default to deny remote access until an administrator actually needs access
• Just in time access must be requested and approved, all of which is audited in the Azure control plane and queryable
via Kusto queries in Log Analytics.
• Once access has been granted, it will be limited to the administrator's IP address and for a set amount of time.
Windows Firewall
• Enable it
• Use it
• Love it
Auditing
• Logs can be stored
• Routed to Azure Log Analytics
• Queried using Kusto
Snapshot the VM
• If you are going to make any major changes to your virtual machine that may have ramifications of taking down your data workloads,
it is important that you utilize virtual machine snapshots in case you want to roll back those changes
• https://docs.microsoft.com/azure/virtual-machines/windows/snapshot-copy-managed-disk
Consider Data Virtualization
Big SQL Database
Move cold
tables
Move Blobs
Parquet Files
T-SQL
Create Table Statement
SELECT *
FROM <table in SQL>
JOIN <table in Azure Blob
Storage>
ON……
Smaller/faster
/cheaper SQL
Database
Ready to
move to
PAAS
Platform Modernization: SSAS
• Azure Analysis Services – PAAS Service
• Only supports tabular
• Doesn’t support multi-dimensional
• Not all data sources available in SSAS are available
• Not all DAX Formulas are supported
• Some limitations with DirectQuery in tabular models
• Migration: You can backup and restore!
• Or re-deploy and reprocess
Platform Modernization SSAS
• Consider it the networking setup between the PaaS Service and the on-premises data sources
• Processing might take longer
• ExpressRoute is still key!
Platform Modernization: SSIS
• Move to Azure Data Factory
• SSIS can either be converted, or run inside of ADF
• Data Migration Assistant can help!
• We saw it on a SQL Server database
• It will work on SSIS, too
• Will flag any migration blockers and highlight recommendations around deprecated features and other potential concerns that may not be blockers
Platform Modernization: SSIS
• Migrate SSIS catalog (SSISDB)
• Can move to Azure SQL Database MI
Platform Modernization: SSRS
• You love it, I love it, now what do we do with it?
• Keep it in a VM
• Deploy new reports to Power BI
• RDL Migration Tool: https://github.com/microsoft/RdlMigration
Platform Modernization: SSRS
• The tool will only migrate the RDL files, the following are not supported:
• Shared data sources
• Shared datasets
• Resources, like image files
• KPIs (SSRS 2016, or later—Enterprise Edition only)
• Mobile reports (SSRS 2016, or later—Enterprise Edition only)
• Report models (deprecated)
• Report parts (deprecated)
but how do we get from here:
huge sql database
app 1
app 2
app 3
excel
ssrs
reports
ssis etl
custom app
data dump to
vendor
sql agent job
that closes the
month
somebody hand made
a lookup table for one
report
linked server for
accounting process
stored procs
on a different
server
email
notification
for alerting
power bi
step 3: now that you’ve de-coupled the database using mi,
let’s see if you can move to paas
much smaller
sql database
app 1
cosmosdb
app 2
much smaller
sql database
app 3
remember this is the goal
how do we get there?
the main problems with legacy applications
• too many dependencies
• bleeding layers
• code that is unreadable
• code that no one wants to change
• repeatedly introducing the same bugs
Michael Feathers
https://www.amazon.com/Working-
Effectively-Legacy-Michael-
Feathers/dp/0131177052
stinting / seam testing
a seam is a place where you can alter behavior in your program without
editing in that place.
we split the method into two methods without changing functionality
this allows us to cover one or both of the methods in test
once covered in tests, we can begin to change functionality
testing strategies
• imagine we’re pulling a puzzle
apart, but leaving the pieces
where they are
• soon we have separate pieces,
but now we can look at them
individually
• move the individual piece to a
paas offering
• can it be deployed separately
only convert that subset to paas
good candidates are:
• archival
• authentication
• notification services
concentrated re-write
only rewrite a small target that you can rip out of the legacy app
let the developers choose their data store
guide them towards the products you know
develop your expertise
remember our main rules
only the application touches the database
all integration is done through an event hub
the bi teams can consume the hub too…..they don’t need to touch the db
what’s a good boundary?
think in terms of deployment
what features are likely to be tested and deployed together
keep the testing surface area small-enough but not too small
deploy the database with the code
version both of them
think in terms of remediation. how are conflicts resolved?
the death of the dba, you are now an azure data architect
• roles and responsibilities
• day to day life
• rise of your career
things you no longer need to do
• sweat about disk space
• worry about patch level and security patches
• create an upgrade plan
• care about what the ops guys are doing
• wonder if you’re going to get priority on the san
• purchase hardware
• argue about capital expenses
things you absolutely have to do
• follow every blog and twitter account for any product you’re responsible for
• Life moves pretty fast. If you don't stop and look around once in a while,
you could miss it.
• learn as much as you can about a wide variety of products
• constantly interact with the developers. it’s your day to watch them.
• de-couple everything from your schema so you can normalize or de-normalize
as necessary
• consider partitioning
• learn scripting because the number of database…
• well, they’re about to explode
day to day life
• create scripts for all manual activities
• learn terraforming
• learn scripting and automation
• create framework to easily deploy dev/test
environments
• learn docker/Kubernetes
• learn python probably
• watch your data model
• watch your performance
• predict scale-up/scale-down
• watch your bill
• watch those developers
rise of your career
• as you embrace the cloud, you will find lots of future opportunities
• no longer is knowledge about only one database product acceptable
• learn no-sql
• learn what mpp and analytics teams are doing
• learn about data pipelining
• double your income
Ike Ellis, MVP
General Manager – Data & AI Practice
Solliance
@ike_ellis
www.ikeellis.com
youtube.com/IkeEllisOnTheMic
ike@solliance.net
• Founder of San Diego
Power BI and PowerApps
UserGroup
• Founder of the San Diego
Software Architecture
Group
• MVP since 2011
• Author of Developing Azure
Solutions, Power BI MVP
Book
• Speaker at PASS Summit,
SQLBits, DevIntersections,
TechEd, Craft, Microsoft
Azure & AI Conference

More Related Content

What's hot

Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...
Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...
Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...Edureka!
 
Introduction to snowflake
Introduction to snowflakeIntroduction to snowflake
Introduction to snowflakeSunil Gurav
 
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...Edureka!
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake OverviewJames Serra
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Databricks
 
Maximum Availability Architecture - Best Practices for Oracle Database 19c
Maximum Availability Architecture - Best Practices for Oracle Database 19cMaximum Availability Architecture - Best Practices for Oracle Database 19c
Maximum Availability Architecture - Best Practices for Oracle Database 19cGlen Hawkins
 
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...Databricks
 
Databricks Delta Lake and Its Benefits
Databricks Delta Lake and Its BenefitsDatabricks Delta Lake and Its Benefits
Databricks Delta Lake and Its BenefitsDatabricks
 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeKent Graziano
 
Data warehousing and business intelligence project report
Data warehousing and business intelligence project reportData warehousing and business intelligence project report
Data warehousing and business intelligence project reportsonalighai
 
DOAG Oracle Database Vault
DOAG Oracle Database VaultDOAG Oracle Database Vault
DOAG Oracle Database VaultStefan Oehrli
 
Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekMark Kromer
 
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data Platforms
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data PlatformsData Engineer's Lunch #81: Reverse ETL Tools for Modern Data Platforms
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data PlatformsAnant Corporation
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Netezza workload management
Netezza workload managementNetezza workload management
Netezza workload managementBiju Nair
 
[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic
[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic
[DSC Europe 22] Overview of the Databricks Platform - Petar ZecevicDataScienceConferenc1
 

What's hot (20)

Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...
Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...
Apache Hadoop Tutorial | Hadoop Tutorial For Beginners | Big Data Hadoop | Ha...
 
Mariadb une base de données NewSQL
Mariadb une base de données NewSQLMariadb une base de données NewSQL
Mariadb une base de données NewSQL
 
PPT on Hadoop
PPT on HadoopPPT on Hadoop
PPT on Hadoop
 
Introduction to snowflake
Introduction to snowflakeIntroduction to snowflake
Introduction to snowflake
 
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...
Hadoop Tutorial | Big Data Hadoop Tutorial For Beginners | Hadoop Certificati...
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake Overview
 
Spark
SparkSpark
Spark
 
Spark architecture
Spark architectureSpark architecture
Spark architecture
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
 
Maximum Availability Architecture - Best Practices for Oracle Database 19c
Maximum Availability Architecture - Best Practices for Oracle Database 19cMaximum Availability Architecture - Best Practices for Oracle Database 19c
Maximum Availability Architecture - Best Practices for Oracle Database 19c
 
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...
Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Im...
 
Databricks Delta Lake and Its Benefits
Databricks Delta Lake and Its BenefitsDatabricks Delta Lake and Its Benefits
Databricks Delta Lake and Its Benefits
 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on Snowflake
 
Data warehousing and business intelligence project report
Data warehousing and business intelligence project reportData warehousing and business intelligence project report
Data warehousing and business intelligence project report
 
DOAG Oracle Database Vault
DOAG Oracle Database VaultDOAG Oracle Database Vault
DOAG Oracle Database Vault
 
Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data Week
 
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data Platforms
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data PlatformsData Engineer's Lunch #81: Reverse ETL Tools for Modern Data Platforms
Data Engineer's Lunch #81: Reverse ETL Tools for Modern Data Platforms
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Netezza workload management
Netezza workload managementNetezza workload management
Netezza workload management
 
[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic
[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic
[DSC Europe 22] Overview of the Databricks Platform - Petar Zecevic
 

Similar to Migrate a successful transactional database to azure

Azure from scratch part 3 By Girish Kalamati
Azure from scratch part 3 By Girish KalamatiAzure from scratch part 3 By Girish Kalamati
Azure from scratch part 3 By Girish KalamatiGirish Kalamati
 
Geek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsGeek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsIDERA Software
 
Tech-Spark: Azure SQL Databases
Tech-Spark: Azure SQL DatabasesTech-Spark: Azure SQL Databases
Tech-Spark: Azure SQL DatabasesRalph Attard
 
Who's in your Cloud? Cloud State Monitoring
Who's in your Cloud? Cloud State MonitoringWho's in your Cloud? Cloud State Monitoring
Who's in your Cloud? Cloud State MonitoringKevin Hakanson
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?James Serra
 
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Eric Bragas
 
Monitorando performance no Azure SQL Database
Monitorando performance no Azure SQL DatabaseMonitorando performance no Azure SQL Database
Monitorando performance no Azure SQL DatabaseVitor Fava
 
SQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsSQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsMichaela Murray
 
Guide to Migration From SQL Server to Azure SQL Database (1).pptx
Guide to Migration From SQL Server to Azure SQL Database (1).pptxGuide to Migration From SQL Server to Azure SQL Database (1).pptx
Guide to Migration From SQL Server to Azure SQL Database (1).pptxlibertyuae uae
 
Designing Microservices
Designing MicroservicesDesigning Microservices
Designing MicroservicesDavid Chou
 
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...gpadmanabh
 
AWS Canberra WWPS Summit 2013 - AWS for Web Applications
AWS Canberra WWPS Summit 2013 - AWS for Web ApplicationsAWS Canberra WWPS Summit 2013 - AWS for Web Applications
AWS Canberra WWPS Summit 2013 - AWS for Web ApplicationsAmazon Web Services
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure Antonios Chatzipavlis
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure Antonios Chatzipavlis
 
Scalable relational database with SQL Azure
Scalable relational database with SQL AzureScalable relational database with SQL Azure
Scalable relational database with SQL AzureShy Engelberg
 
SQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsSQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsMichaela Murray
 

Similar to Migrate a successful transactional database to azure (20)

Azure from scratch part 3 By Girish Kalamati
Azure from scratch part 3 By Girish KalamatiAzure from scratch part 3 By Girish Kalamati
Azure from scratch part 3 By Girish Kalamati
 
Geek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsGeek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure Environments
 
Tech-Spark: Azure SQL Databases
Tech-Spark: Azure SQL DatabasesTech-Spark: Azure SQL Databases
Tech-Spark: Azure SQL Databases
 
AZURE Data Related Services
AZURE Data Related ServicesAZURE Data Related Services
AZURE Data Related Services
 
Who's in your Cloud? Cloud State Monitoring
Who's in your Cloud? Cloud State MonitoringWho's in your Cloud? Cloud State Monitoring
Who's in your Cloud? Cloud State Monitoring
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?
 
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Intro to Azure Data Factory v1
Intro to Azure Data Factory v1
 
Monitorando performance no Azure SQL Database
Monitorando performance no Azure SQL DatabaseMonitorando performance no Azure SQL Database
Monitorando performance no Azure SQL Database
 
SQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsSQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT Solutions
 
Guide to Migration From SQL Server to Azure SQL Database (1).pptx
Guide to Migration From SQL Server to Azure SQL Database (1).pptxGuide to Migration From SQL Server to Azure SQL Database (1).pptx
Guide to Migration From SQL Server to Azure SQL Database (1).pptx
 
Migrate SQL Workloads to Azure
Migrate SQL Workloads to AzureMigrate SQL Workloads to Azure
Migrate SQL Workloads to Azure
 
Designing Microservices
Designing MicroservicesDesigning Microservices
Designing Microservices
 
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...
VMworld 2013: Vapp6124 automating v mware cloud and virtualization deployment...
 
AWS Canberra WWPS Summit 2013 - AWS for Web Applications
AWS Canberra WWPS Summit 2013 - AWS for Web ApplicationsAWS Canberra WWPS Summit 2013 - AWS for Web Applications
AWS Canberra WWPS Summit 2013 - AWS for Web Applications
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Azure SQL Database
Azure SQL DatabaseAzure SQL Database
Azure SQL Database
 
A to z for sql azure databases
A to z for sql azure databasesA to z for sql azure databases
A to z for sql azure databases
 
Scalable relational database with SQL Azure
Scalable relational database with SQL AzureScalable relational database with SQL Azure
Scalable relational database with SQL Azure
 
SQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT SolutionsSQL Server 2019 hotlap - WARDY IT Solutions
SQL Server 2019 hotlap - WARDY IT Solutions
 

More from Ike Ellis

Storytelling with Data with Power BI
Storytelling with Data with Power BIStorytelling with Data with Power BI
Storytelling with Data with Power BIIke Ellis
 
Storytelling with Data with Power BI.pptx
Storytelling with Data with Power BI.pptxStorytelling with Data with Power BI.pptx
Storytelling with Data with Power BI.pptxIke Ellis
 
Build a modern data platform.pptx
Build a modern data platform.pptxBuild a modern data platform.pptx
Build a modern data platform.pptxIke Ellis
 
Data Modeling on Azure for Analytics
Data Modeling on Azure for AnalyticsData Modeling on Azure for Analytics
Data Modeling on Azure for AnalyticsIke Ellis
 
Data modeling trends for analytics
Data modeling trends for analyticsData modeling trends for analytics
Data modeling trends for analyticsIke Ellis
 
Data modeling trends for Analytics
Data modeling trends for AnalyticsData modeling trends for Analytics
Data modeling trends for AnalyticsIke Ellis
 
Relational data modeling trends for transactional applications
Relational data modeling trends for transactional applicationsRelational data modeling trends for transactional applications
Relational data modeling trends for transactional applicationsIke Ellis
 
Power bi premium
Power bi premiumPower bi premium
Power bi premiumIke Ellis
 
Move a successful onpremise oltp application to the cloud
Move a successful onpremise oltp application to the cloudMove a successful onpremise oltp application to the cloud
Move a successful onpremise oltp application to the cloudIke Ellis
 
Azure Databricks is Easier Than You Think
Azure Databricks is Easier Than You ThinkAzure Databricks is Easier Than You Think
Azure Databricks is Easier Than You ThinkIke Ellis
 
Pass 2018 introduction to dax
Pass 2018 introduction to daxPass 2018 introduction to dax
Pass 2018 introduction to daxIke Ellis
 
Pass the Power BI Exam
Pass the Power BI ExamPass the Power BI Exam
Pass the Power BI ExamIke Ellis
 
Slides for PUG 2018 - DAX CALCULATE
Slides for PUG 2018 - DAX CALCULATESlides for PUG 2018 - DAX CALCULATE
Slides for PUG 2018 - DAX CALCULATEIke Ellis
 
Introduction to DAX
Introduction to DAXIntroduction to DAX
Introduction to DAXIke Ellis
 
60 reporting tips in 60 minutes - SQLBits 2018
60 reporting tips in 60 minutes - SQLBits 201860 reporting tips in 60 minutes - SQLBits 2018
60 reporting tips in 60 minutes - SQLBits 2018Ike Ellis
 
14 Habits of Great SQL Developers
14 Habits of Great SQL Developers14 Habits of Great SQL Developers
14 Habits of Great SQL DevelopersIke Ellis
 
14 Habits of Great SQL Developers
14 Habits of Great SQL Developers14 Habits of Great SQL Developers
14 Habits of Great SQL DevelopersIke Ellis
 
Dive Into Azure Data Lake - PASS 2017
Dive Into Azure Data Lake - PASS 2017Dive Into Azure Data Lake - PASS 2017
Dive Into Azure Data Lake - PASS 2017Ike Ellis
 
A lap around microsofts business intelligence platform
A lap around microsofts business intelligence platformA lap around microsofts business intelligence platform
A lap around microsofts business intelligence platformIke Ellis
 
Survey of the Microsoft Azure Data Landscape
Survey of the Microsoft Azure Data LandscapeSurvey of the Microsoft Azure Data Landscape
Survey of the Microsoft Azure Data LandscapeIke Ellis
 

More from Ike Ellis (20)

Storytelling with Data with Power BI
Storytelling with Data with Power BIStorytelling with Data with Power BI
Storytelling with Data with Power BI
 
Storytelling with Data with Power BI.pptx
Storytelling with Data with Power BI.pptxStorytelling with Data with Power BI.pptx
Storytelling with Data with Power BI.pptx
 
Build a modern data platform.pptx
Build a modern data platform.pptxBuild a modern data platform.pptx
Build a modern data platform.pptx
 
Data Modeling on Azure for Analytics
Data Modeling on Azure for AnalyticsData Modeling on Azure for Analytics
Data Modeling on Azure for Analytics
 
Data modeling trends for analytics
Data modeling trends for analyticsData modeling trends for analytics
Data modeling trends for analytics
 
Data modeling trends for Analytics
Data modeling trends for AnalyticsData modeling trends for Analytics
Data modeling trends for Analytics
 
Relational data modeling trends for transactional applications
Relational data modeling trends for transactional applicationsRelational data modeling trends for transactional applications
Relational data modeling trends for transactional applications
 
Power bi premium
Power bi premiumPower bi premium
Power bi premium
 
Move a successful onpremise oltp application to the cloud
Move a successful onpremise oltp application to the cloudMove a successful onpremise oltp application to the cloud
Move a successful onpremise oltp application to the cloud
 
Azure Databricks is Easier Than You Think
Azure Databricks is Easier Than You ThinkAzure Databricks is Easier Than You Think
Azure Databricks is Easier Than You Think
 
Pass 2018 introduction to dax
Pass 2018 introduction to daxPass 2018 introduction to dax
Pass 2018 introduction to dax
 
Pass the Power BI Exam
Pass the Power BI ExamPass the Power BI Exam
Pass the Power BI Exam
 
Slides for PUG 2018 - DAX CALCULATE
Slides for PUG 2018 - DAX CALCULATESlides for PUG 2018 - DAX CALCULATE
Slides for PUG 2018 - DAX CALCULATE
 
Introduction to DAX
Introduction to DAXIntroduction to DAX
Introduction to DAX
 
60 reporting tips in 60 minutes - SQLBits 2018
60 reporting tips in 60 minutes - SQLBits 201860 reporting tips in 60 minutes - SQLBits 2018
60 reporting tips in 60 minutes - SQLBits 2018
 
14 Habits of Great SQL Developers
14 Habits of Great SQL Developers14 Habits of Great SQL Developers
14 Habits of Great SQL Developers
 
14 Habits of Great SQL Developers
14 Habits of Great SQL Developers14 Habits of Great SQL Developers
14 Habits of Great SQL Developers
 
Dive Into Azure Data Lake - PASS 2017
Dive Into Azure Data Lake - PASS 2017Dive Into Azure Data Lake - PASS 2017
Dive Into Azure Data Lake - PASS 2017
 
A lap around microsofts business intelligence platform
A lap around microsofts business intelligence platformA lap around microsofts business intelligence platform
A lap around microsofts business intelligence platform
 
Survey of the Microsoft Azure Data Landscape
Survey of the Microsoft Azure Data LandscapeSurvey of the Microsoft Azure Data Landscape
Survey of the Microsoft Azure Data Landscape
 

Recently uploaded

QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdfHulkTheDevil
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfROWELL MARQUINA
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerAnchore
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactivestartupro
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MIRomil Mishra
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionDEEPRAJ PATHAK
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxKunal Gupta
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Memoori
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfwill854175
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 

Recently uploaded (20)

QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdf
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey Hightower
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactive
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MI
 
BoSEU24 | Bill Thompson | Talk From Another Century
BoSEU24 | Bill Thompson | Talk From Another CenturyBoSEU24 | Bill Thompson | Talk From Another Century
BoSEU24 | Bill Thompson | Talk From Another Century
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile Evolution
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptx
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdf
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 

Migrate a successful transactional database to azure

  • 1. Move a Successful SQL Server Transactional Database to Azure
  • 2. Ike Ellis, MVP General Manager – Data & AI Practice Solliance @ike_ellis www.ikeellis.com youtube.com/IkeEllisOnTheMic • Founder of San Diego Power BI and PowerApps UserGroup • Founder of the San Diego Software Architecture Group • MVP since 2011 • Author of Developing Azure Solutions, Power BI MVP Book • Speaker at PASS Summit, SQLBits, DevIntersections, TechEd, Craft, Microsoft Azure & AI Conference
  • 3. Here’s what you get in this course • All the slide decks • Videos of every presentation that you can watch forever, share with friends, do what you want with it • Lab #1: Hybrid Azure/On-premise AlwaysOn Ags • https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually- configure-prerequisites-tutorial • Lab #2: Microsoft Cloud Workshop • https://microsoftcloudworkshop.com/ • Migrate SQL Server to Azure • https://github.com/microsoft/MCW-Migrating-SQL-databases-to-Azure • Migrate Oracle to Azure (Postgres or Azure SQL) • https://github.com/Microsoft/MCW-Migrating-Oracle-to-Azure-SQL-and-PostgreSQL
  • 4. Microsoft Migration Guides • https://docs.microsoft.com/en-us/data-migration/
  • 5. Agenda For The Course • Introduction to Azure database options for SQL Server migrations • Migration planning and introduction to migration techniques • Migration preparation • Migration to IAAS Azure VMs • Migration to PAAS - Azure SQL Managed Instance • Migration to PAAS - Azure SQL Database • Post migration cleanup and migrating other things • Platform modernization • Call to action
  • 6. Do you have something like this? huge SQL database app 1 app 2 app 3
  • 7. But really it’s like this: huge SQL database app 1 app 2 app 3 Excel SSRS reports SSIS etl custom app data dump to vendor SQL Agent job that closes the month somebody hand made a lookup table for one report linked server for accounting process stored procs on a different server email notification for alerting Power BI
  • 8. Not a cloud app: reason #1: too expensive • In the cloud, you pay for the following • compute • network • disk • memory • But the most expensive thing by far: • compute • When the database is that large, you are paying for a lot of compute for a database that is essentially hosting a lot of data at rest
  • 9. Not a cloud app: reason #2: too expensive huge SQL database scales up as a unit. this is so big that if we need more power, we have to scale up the entire huge thing this tiny piece here is really important and should scale separately
  • 10. Not a cloud app: reason #3: bad messaging huge SQL database app 1 app 2 this is an expensive and slow message bus that takes a lot of grooming. you’re getting overcharge for storage and compute
  • 11. Not a cloud app: reason #4: too hard to change huge sql database app 1 app 2 app 3 Excel SSRS reports SSIS etl custom app data dump to vendor SQL Agent job that closes the month somebody hand made a lookup table for one report linked server for accounting process stored procs on a different server email notification for alerting Power BI in order to change it, look at how many things have to be tested and verified and deployed together
  • 12. not a cloud app: reason #4: too hard to deploy huge SQL database app 1 app 2 app 3 Excel SSRS reports SSIS etl custom app data dump to vendor SQL Agent job that closes the month somebody hand made a lookup table for one report linked server for accounting process stored procs on a different server email notification for alerting Power BI in order to deploy this, look at how many things have to be tested and verified and deployed together
  • 13. in order to upgrade this to a new version of sql, look at how many things have to be tested and verified and upgraded together Not a cloud app: reason #4: too hard to upgrade huge SQL database app 1 app 2 app 3 Excel SSRS reports SSIS etl custom app data dump to vendor SQL Agent job that closes the month somebody hand made a lookup table for one report linked server for accounting process email notification for alerting Power BI
  • 14. Reason #5: not secure huge SQL database pii data all up in this
  • 15. Reason #6: all kinds of performance problems huge sql database For far too many years the answer to what to do to solve all kinds of performance problems was: Throw hardware at it until it goes away Now we have 32 cores with an average cpu utilization of 2% The cloud will punish you for decisions like this
  • 16. Simple example of a cloud-scale application much smaller SQL database app 1 CosmosDB app 2 much smaller SQL database app 3
  • 17. How do they integrate? much smaller SQL database app 1 CosmosDB app 2 much smaller SQL database app 3
  • 18.
  • 19. Azure SQL Options • Azure SQL in a VM • Azure SQL Database • Single database • Azure SQL Database Managed Instance • Cross between SQL in a VM and Azure SQL Database • Azure SQL Elastic Pool • Elastic pools are a collection of databases that share a set of resources, providing an efficient, cost-effective solution for managing multiple databases that have variable usage patterns • Azure SQL Serverless
  • 20. Azure SQL Database Options for Migration Azure SQL DB
  • 21. A recommended migration path On-premise SQL Server Azure VM SQL Server
  • 22. Planning for an On-Premise SQL Server to Migrate to Azure Ike Ellis, MVP Solliance General Manager – Data & AI Practice
  • 23. Cloud Adoption Framework • https://docs.microsoft.com/azure/cloud-adoption-framework/ • Rehost: This path involves the replacement of the old hosting environment with a new one, such as the Azure cloud, with minimal changes to the architecture. • Refactor: With this path, modifications are made to applications and databases to allow for the use of PaaS-based services. • Rearchitect: This involves updating the parts of an application or database that are incompatible with moving to the cloud. Frequently, this is due to past decisions made when initially building the applications and databases. • Rebuild: This path involves rebuilding the app from scratch to support cloud-native concepts. • Replace: If an app cannot move without significant time, effort, and expense, it may be time to replace the app with a new one that is more cloud friendly such as a Software-as-a-Service (SaaS) based one.
  • 24. Azure database migration journey Assess Optimize Migrate Enables optimization during or post migration (fully managed service) IaaS (virtual machines) fall short here Enables rehosting or light refactoring for most apps Eliminates the need to rearchitect or rebuild your apps SQ L
  • 25. Document and design your Azure management strategy • Manage resources: Understand best practices for Azure resource groups and resources, including smart naming, preventing accidental deletion, managing resource permissions, and effective resource tagging. • Use blueprints: Get a quick overview on using blueprints for building and managing your deployment environments. • Set up management groups: If you have multiple subscriptions, you can gather them into management groups, and apply governance settings to those groups. • Set up access policies: Apply compliance policies to your Azure resources. • Design and, if practicable, test, a recovery plan: Put together a business continuity and disaster recovery (BCDR) strategy to keep data safe, your environment resilient, and resources up and running when outages occur. • Manage VMs: Group VMs into availability groups for resilience and high availability. Use managed disks for ease of VM disk and storage management. • Monitor resource usage: Enable diagnostic logging for Azure resources, build alerts and playbooks for proactive troubleshooting, and use the Azure dashboard for a unified view of your deployment health and status. • Manage support and updates: Understand your Azure support plan and how to implement it, get best practices for keeping VMs up-to-date, and put processes in place for change management. • Review architectures: Review sample Azure architectures to learn from as you build your post- migration deployments.
  • 26. Choose your cloud type • Azure public Cloud • Hybrid cloud • Azure Government Cloud
  • 27. Document your current on-premise architecture Document how each component will go to the cloud
  • 28. Reasons to go to IAAS first – Look at what you get! • No hardware to buy, manage, maintain and ultimately retire • No management of the any supporting infrastructure • Fast setup of environment and quick scaling • Automatic upgrades, patching, and updates • Integration with other services • Built-in high-availability • Automated backups • Money-backed Service Level Agreements (SLA)
  • 29. Things you will have to change when you go to IAAS • Network configuration • Installation of agents • Different vCPU types • Security systems • Authentication methods • SQL-based HADR
  • 30. When to upgrade to SQL Server 2019 • Upgrade first • Migrate to Azure • Or Migrate to Azure • Upgrade But do not upgrade and migrate at the exact same time Too much change. Too many variables. Too hard to rollback
  • 31. What we learn when we migrate to IAAS first • We’ve solved • Networking • Security • Data migration • We’ve learned • Everything that was affected by doing that, including: • What apps connected to it • If those apps have the needed network latency • Excel • SSIS • SSRS • And we’ve catalogued and documented everything that we had to touch while doing the simple path first
  • 32. Plan your licensing in the VM and Azure SQL DB MI and Azure SQL DB • Pay-as-you-go licensing allows businesses to use SQL Server images in Azure on a pay-as-you-go basis, rather than requiring license keys. • Bring-your-own-license (BYOL) is a model that allows organizations to take advantage of SQL Server licenses they already own. Running SQL Server in Azure can often provide even deeper discounts on licensing. • Azure Hybrid Use Benefits may be available to companies with an existing Enterprise Agreement (EA). To learn more, check out the Azure Hybrid Benefits offering.
  • 33. Reserve azure sql db resources in advance and save up to 33%1 • budget and forecast better with upfront payment for one-year or three-year terms • get prioritized compute capacity in azure regions • exchange or cancel reservations as your needs evolve • scale up or down within a performance tier and region with auto-fit • move saas apps between elastic pools and single databases and keep your reserved instance benefit Reserved capacity for azure sql db License included With Azure Hybrid Benefit Up to 55% savings2 Reserved Instances with Azure Hybrid Benefit Up to 80% savings3
  • 34. • Discounted rates up to 55% off to support your ongoing development and testing • Dev/test pricing available for vcore- based deployment options • Eligible with active visual studio subscription • Your Visual Studio subscription comes with $150/month of free Azure Dev/test pricing for sql db
  • 35. Most important! Hire your own Tim!
  • 36. Prepare to Migrate an on-premise SQL Server to Azure Ike Ellis Solliance General Manager – Data + AI Practice Microsoft MVP
  • 37. First things first: let’s pay off some technical debt • before you can begin a cloud migration of a database, here are some great things you can do to get ready for the project
  • 38. Decouple from the server name • Create a DNS CName (alias) for your sql server • Change the ConnectionString for all applications, Excel, SSIS, SSRS, Power BI, linked servers, backup tools, SSMS, SSAS, and everything else you can think of • Make a list of every single location where you had to make the change • This way when the server changes location, you can just change the cname • If you really want to see if you succeeded at this or not, rename the server • https://docs.microsoft.com/en-us/sql/database-engine/install-windows/rename-a- computer-that-hosts-a-stand-alone-instance-of-sql-server?view=sql-server-ver15 App 1 DNS SQL Server DB Connect
  • 39. Decouple database names • Look for three part object names in stored procedures in this database and other databases • worldwideimporters.dbo.customers
  • 40. Make the database smaller • Every time I ask developers or business users if we can delete some data, the answer is always no • When we actually look at the data, there is plenty we can either delete or move • Really old data • Audit logs and audit records • Temp tables • Tables that are empty or have really old data in them or rarely used data • Reporting tables that really should be in a data warehouse • Indexes that no one ever uses • Remove overlapping indexes
  • 41. Find unused indexes SELECT objects.name AS Table_name, indexes.name AS Index_name, dm_db_index_usage_stats.user_seeks, dm_db_index_usage_stats.user_scans, dm_db_index_usage_stats.user_updates FROM sys.dm_db_index_usage_stats INNER JOIN sys.objects ON dm_db_index_usage_stats.OBJECT_ID = objects.OBJECT_ID INNER JOIN sys.indexes ON indexes.index_id = dm_db_index_usage_stats.index_id AND dm_db_i ndex_usage_stats.OBJECT_ID = indexes.OBJECT_ID WHERE AND dm_db_index_usage_stats.user_lookups = 0 AND dm_db_index_usage_stats.user_seeks = 0 AND dm_db_index_usage_stats.user_scans = 0 ORDER BY dm_db_index_usage_stats.user_updates DESC
  • 42. Find overlapping indexes select s.Name + N'.' + t.name as [Table] ,i1.index_id as [Index1 ID], i1.name as [Index1 Name] ,dupIdx.index_id as [Index2 ID], dupIdx.name as [Index2 Name] ,c.name as [Column] from sys.tables t join sys.indexes i1 on t.object_id = i1.object_id join sys.index_columns ic1 on ic1.object_id = i1.object_id and ic1.index_id = i1.index_id and ic1.index_column_id = 1 join sys.columns c on c.object_id = ic1.object_id and c.column_id = ic1.column_id join sys.schemas s on t.schema_id = s.schema_id cross apply ( select i2.index_id, i2.name from sys.indexes i2 join sys.index_columns ic2 on ic2.object_id = i2.object_id and ic2.index_id = i2.index_id and ic2.index_column_id = 1 where i2.object_id = i1.object_id and i2.index_id > i1.index_id and ic2.column_id = ic1.column_id ) dupIdx order by s.name, t.name, i1.index_id if the leading edge is the same, the indexes can likely be combined
  • 43. Implement filtered indexes My feeling is that every index should be filtered for legacy databases that have been around for more than 3 years create nonclustered index IDX_Data_Unprocessed_Filtered on dbo.Data(RecId) include(Processed) where Processed = 0;
  • 44. Make the database smaller: defrag and shrink • Many of us have databases that have not been properly maintained • Defrag! • Get rid of the free space in datafiles • Find the worst offenders: select f.type_desc as [Type] ,f.name as [FileName] ,fg.name as [FileGroup] ,f.physical_name as [Path] ,f.size / 128.0 as [CurrentSizeMB] ,f.size / 128.0 - convert(int,fileproperty(f.name,'SpaceUsed')) / 128.0 as [FreeSpaceMb] from sys.database_files f with (nolock) left outer join sys.filegroups fg with (nolock) on f.data_space_id = fg.data_space_id option (recompile)
  • 45. Make the database smaller: compression • Introduce page compression • Columnstore indexing
  • 46. Make the database smaller: get the blobs out • Blobs don’t belong in a database • Move them to azure blob storage first! • C# is probably how you got them in there, write a C# app to get them out and move them to abs • Save the URL to the file in the database as a varchar(100) • Change the application to pull from abs instead of the database • Benefits of getting the blobs • Significantly reduces the size of the database • Backed up automatically • Much cheaper to save those files in abs than pay to have azure SQL database store them • Consistent load time no matter how many there are in there. SQL can’t say that • With ABS we don’t pay for compute time! just storage! • If you don’t do this, at least compress them where they are, but that’s a similar amount of work
  • 47. Find bad actors with the network • Use profiler, extended events, wireshark, fiddler, chrome network tab • Look for bad actors who are using too much network • It’s easy to find pages that are doing the exact same code twice • Or have n+1 problems • Or are bringing back too much data and not paging it • Saving the network will make your app more cloud-ready
  • 48. Before the cloud migration, move your backup/restore process to azure • If you’re currently backing up your databases to an on-premise location, change that process to move it to azure blob storage • This is a great way to actually get your database to the cloud in BAK form • Azure blob storage can be both a backup destination and a restore source!
  • 49. Consider an on-premise upgrade of sql server • Upgrades will get you on the latest version of the sql server optimizer and will prepare you to use an azure sql database version of the optimizer • Will give you features for compression and data management that you might not have right now
  • 50. Download some things • Download and install the latest version of the MAP Toolkit. • provides a readiness report to migrate the server • Download and install the latest version of the Database Experimentation Assistant • Download and install the Data Migration Assistant v3.3 or later • Create an instance of sql server on Azure VMs by following the detail in the article How to provision a Windows SQL Server virtual machine in the Azure portal
  • 52.
  • 53.
  • 54. Take care of azure active directory https://docs.microsoft.com/en-us/azure/active-directory/
  • 55. Options for extending Active Directory • AD Connect from a machine that’s on prem and that connects to azure active directory • Active Directory domain controller on a vm • Create groups in ad on prem that map to what you want to secure in azure • Add rights for resource groups • Create simple groups on prem in ad that are the exact names of the resource groups in your ad subscription
  • 56. Why are we thinking about VMs? • Even if you aren’t using IAAS, you will still need VMs • Somewhere for SSMS/ADS • Somewhere to move data around without getting charged ingress/egress • Somewhere where the latency is native and close to the data center • And you might go PAAS for the database, but go IAAS for SSRS, SSIS, SQL Server Agent • Networking can be a challenge • Opening ports should be done sparingly • A jump box on your VNet will let you manage Azure SQL DB MI • You can pause it to stop getting billed for it
  • 57. Disks • SQL is I/O bound. I/O is incredibly important
  • 58. It’s all about the disks (i/o) baby Disk options: https://azure.microsoft.com/en-us/pricing/details/managed-disks/ • Premium SSD managed disks • high performance, managed disks • SSD
  • 59. Ultra disks not available in all regions
  • 60. Storage space: ack! • Stay away from storage spaces • Taxes the cpu and that’s expensive • Doesn’t really give you more IOPS
  • 62. azure VM: other disk recommendations • Remember premium ssd or ultra ssd. • Standard storage is only recommended for dev/test. • Keep the storage account and vm in the same region. • Disable azure geo-redundant storage (geo-replication) on the storage account. • Azure disk striping for performance
  • 63. remember your disk best practices • Use a minimum of 2 p30 disks (1 for log files and 1 for data files including tempdb). • For workloads requiring ~50,000 IOPS, consider using an Ultra SSD. • Avoid using operating system or temporary disks for database storage or logging. • Enable read caching on the disk(s) hosting the data files and tempdb data files. • Do not enable caching on disk(s) hosting the log file. • important: Stop the SQL Server service when changing the cache settings for an azure vm disk. • Stripe multiple azure data disks to get increased IO throughput • but be careful! you will load the cpu here! • Format with documented allocation sizes. • Place TempDB on the local ssd d: drive for mission critical sql server workloads (after choosing correct VM size).
  • 64. Some common things to watch out for in source systems • Single or dual disk SQL Server configuration where data and logs are mixed onto single disk; you should explore breaking these apart. • Single disk (operating system disk with data and log files) that is greater than 2TB • Having a set of databases in which sum of data file size is greater than 32TB, you will need to break them apart into separate disks for Azure Managed Premium Disk support or you will be required to use the Ultra Disk option that has limits up to 64TB. • A single data or log file inside a managed disk that is greater than 4TB • If you have more than 2PB of disk storage, you will need to break out your database workloads onto multiple servers; or consider moving to a PaaS • IOPS greater than 20,000 on single server instance; consider clustering • Throughput higher than 900MB/sec
  • 65. Choose the right VM size • Watch the cpu/memory ratio • Use Azure Migrate to determine hardware requirements • Test them all! • Not all types are available in all Azure regions, so choose your region carefully
  • 66. The importance of a performance baseline Create a baseline, but also know what load your hardware can handle
  • 67. Migrate an on-premise SQL Server to IAAS – Azure VMs Ike Ellis Solliance General Manager – Data + AI Practice Microsoft MVP
  • 68. Agenda • Demo: Configure the VM • Configure TempDB • Demo backup and restore • This is easier if you are already backing up and restoring to an Azure Blob Storage URL • Demo: Backup/restore to URLs • Demo: Migrating SQL Agent Jobs • Demo: Migrating Logins • Hybrid-cloud Azure AGs
  • 69. Availability Zone • Minimum of three separate zones in all enabled regions • Each zone has it’s own power, cooling, and networking • Zone-redundant services replicate your applications and data across Availability Zones to protect from single-points-of-failure • If you create three or more VMs across three zones in an Azure region, your VMs are effectively distributed across three fault domains and three update domains
  • 70. Availability Set • Ensures that each VM is in it’s own rack and it’s own update domain • https://docs.microsoft.com/en-us/azure/virtual-machines/manage-availability
  • 71. Configure TempDB • Use Mike Petri’s script • https://gist.github.com/FlogDonkey/3b9851f1c13b94f3f2d26b4ab5b49de0
  • 73. Demo: Backup/Restore to Azure Blob Storage
  • 74. Demo: Migrating SQL Agent Jobs
  • 75. Demo: Migrating logins • https://support.microsoft.com/en-us/help/918992/how-to-transfer-logins-and-passwords-between-instances-of-sql-server
  • 76. No downtime migration: Azure AlwaysOn AGs On-premise primary + Azure secondaries • https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually- configure-prerequisites-tutorial • https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/availability-group-manually- configure-tutorial
  • 77. Who is Azure SQL Database Managed Instance for? customers looking to migrate a large number of apps from on-premise or iaas, self-built or isv provided, with as low migration effort as possible & cost being a crucial factor
  • 78. • Enable full isolation from other tenants without resource sharing • Promote secure communication over private IP addresses with native vnet integration • Enable your on-premise identities on cloud instances, through integration with azure active directory and ad connect • Combine the best of SQL Server with the benefits of a fully-managed service • Use familiar SQL server features in SQL database managed instance What is the advantage of azure SQL DB MI over a VM? VNET support in SQL Database Managed Instance
  • 79. Migration to PAAS Azure SQL Database Managed Instance Ike Ellis Solliance General Manager – Data + AI Practice Microsoft MVP
  • 80. Put your DBs on autopilot and focus on your business… * - features coming soon mi has it built-in compute & storage provisioned on demand fast & online scaling full stack updates and patches backups with health checks point-in-time restore (configurable retention *) 99.99% availability with automatic failover disaster recovery with single geo secondary (multiple*)
  • 81. It’s much easier with the mi compliance with all major industry standards threat detection with automatic alerting intelligent query processing automatic performance tuning* monitoring at scale with intelligent insights data discovery and classification* vulnerability assessment Put your DBs on autopilot and focus on your business… * - features coming soon
  • 82. • Fully-fledged SQL instance with nearly 100% compat with on- premise • Built on the same infrastructure as SQL Database • Provides the same benefits (PaaS) • Contained within your VNet • Private IP addresses • Express Route / VPN connectivity SQL Database (PaaS) Elastic Pool Managed Instance Single database • Transparent • Frictionless • Competitive
  • 83. cross-db queries & transactions, linked servers to sql .net clr modules service broker change data capture transactional replication Easily migrate from SQL Server & modernize * - features coming soon choice of instance collations* and instance time zone* R services* msdtc for distributed transactions filestream / filetable, polybase
  • 84. dmvs, xevents, query store SQL agent and DB mail sysadmin privileges and resource governor Easily migrate from SQL Server & modernize SQL auditing, rls TDE, always encrypted, and dynamic data masking built-in ha replaces on-prem setups replace MDW with OMS monitoring network security with vnets and private IPs integrated auth. with azure ad
  • 86. mi: service tiers Capability Service tier General Purpose (GA) Business Critical (Public Preview) Best for Apps with typical availability and common IO latency requirements Apps with highest availability and lowest IO latency requirements. Compute (vCores) 8, 16, 24, 32, 40, 64, 80 8, 16, 24, 32, 40, 64, 80 HA / Recovery Time Objective Remote storage based / Good Always On AG based / Better Storage type / size Fast remote (Azure Premium) / Up to 8 TB Super-fast local SSD / Up to 4 TB Read scale out (read-only replica) No Yes In-Memory OLTP No Yes Price competitive with AWS? Yes, ~33% lower (license included) Yes, ~46% lower (license included)
  • 87. General Purpose Feature Description Number of vCores* 8, 16, 24 (Gen 4) 8, 16, 24, 32, 40, 64, 80 (Gen 5) SQL Server version / build SQL Server (latest available) Min storage size 32 GB Max storage size 8 TB Max storage per database Determined by the max storage size per instance Expected storage IOPS 500-7500 IOPS per data file (depends on data file). See Premium Storage Number of data files (ROWS) per the database Multiple Number of log files (LOG) per database 1 Managed automated backups Yes HA Based on remote storage and Azure Service Fabric Built-in instance and database monitoring and metrics Yes Automatic software patching Yes VNet - Azure Resource Manager deployment Yes VNet - Classic deployment model No Portal support Yes
  • 88. Business Critical Feature Description Number of vCores* 8, 16, 24, 32 (Gen 4) 8, 16, 24, 32, 40, 64, 80 (Gen 5) SQL Server version / build SQL Server (latest available) Additional features In-Memory OLTP 1 additional read-only replica (Read Scale-Out) Min storage size 32 GB Max storage size •Gen 4: 1 TB (all vCore sizes Gen 5:1 TB for 8, 16 vCores •2 TB for 24 vCores •4 TB for 32, 40, 64, 80 vCores Max storage per database Determined by the max storage size per instance Number of data files (ROWS) per the database Multiple Number of log files (LOG) per database 1 Managed automated backups Yes HA Based on Always On Availability Groups and Azure Service Fabric Built-in instance and database monitoring and metrics Yes Automatic software patching Yes VNet - Azure Resource Manager deployment Yes VNet - Classic deployment model No Portal support Yes Business Critical service tier: collocated compute and storage Primary endpoint (read-write) Read-only endpoint Always On AG SQL SQL SQL SQL Super-fast SSD Primary replica Secondary replica Secondary replica Secondary replica
  • 89. SQL DB deployment model overview Azure SQL Database Unit of Monetization Tiering Pricing vs. Competitors  Basic: designed for apps with light workloads  Standard: mid-level performance and business continuity  Premium: low IO latency workloads and higher business continuity  General Purpose  “Business Critical” Hybrid Benefits DTU – “Database Throughput Unit” – measure of database performance that blends CPU, memory and I/O.  vCore for compute  GBs for storage  IOPs for IO Yes, EE customers also get 4 cores in General Purpose SKU  Basic – very cheap because it priced to accommodate web customers  Standard – comparable pricing but not easily explainable to customer  Premium – expensive due to additional replicas and IOs  Priced lower compared to AWS  No Best for New apps, with a ‘one database per app pattern’ and resources guaranteed at DB level Modernizing large number of existing SQL Server apps from on-premises or IaaS New SaaS apps or modernizing existing apps to SaaS, resource sharing across DBs of existing LOB apps for higher efficiency eDTU – elastic “Database Throughput Unit” – measure of database performance that blends CPU, memory and I/O.
  • 90. Deployment resources portal deployment walk-through https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-create-tutorial-portal https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-vnet-configuration arm template deployment walk-through https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/07/02/deploy-azure-sql-managed-instance-network- environment-using-arm/ https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/05/15/creating-azure-sql-managed-instance-using- arm-templates/
  • 91. Be empty: Have specific route table: Optional custom DNS: No service endpoint: Sufficient IP addresses: Virtual network considerations https://docs.microsoft.com/en-us/azure/sql-database/sql-database-managed-instance-vnet-configuration#requirements
  • 92. An MI instance must be deployed in an Azure virtual network • allows for connecting directly from an on-premises network • allows for connecting linked servers or other on-premises data stores • allows for connecting to additional azure resources Plan your deployment • managed instance requires a minimum of 16 IP addresses in a subnet and may use up to 256 IP addresses • if deploying multiple managed instances inside the subnet, you need to optimize the subnet size • the default values create a subnet that takes all the vnet address space, allowing for only mi inside the virtual network Routes • Effective routes on the Managed Instance subnet are not supported • Routes can be user-defined (UDR) or Border Gateway Protocol (BGP) routes propagated to network interfaces through ExpressRoute or site-to-site VPN connections • For BGP routes, create a 0.0.0.0/0 Next Hop Internet route and apply it to the Managed Instance subnet Network security groups (nsg) NSGs on the Managed Instance subnet are not supported Virtual network guidance
  • 93. Considerations when creating a new virtual network for mi • Calculate the subnet size • Assess the needs for the rest of the vnet • Disable service end points • Create new arm virtual network Virtual network guidance Name Any valid name Address space Any valid address range, such as 10.14.0.0/24 Subscription Your subscription Resource Group Any valid resource group (new or existing) Location Any valid location Subnet name Any valid subnet name, such as mi_subnet Subnet address range Any valid subnet address, such as 10.14.0.0/28. Use a subnet address space smaller than the address space itself to allow space to create other subnets in the same VNet, such as a subnet for hosting test / client apps or gateway subnets to connect from on- prem or other VNets. Service endpoints Disabled
  • 94. Built-in high availability P S S Write Write Ack Ack Read write Ack value DB Availability group
  • 95. Surface area of mi •always on the latest and greatest SQL engine version •your code can be SQL deployment model aware if necessary MI is always on latest and greatest SQL engine version documentation page Your code can be SQL deployment model aware if necessary
  • 96. DB compatibility based certification •Microsoft database compatibility level protection •Easy to use tools to help you access migration Microsoft Database Compatibility Level Protection Overall process Contact Microsoft
  • 97. App compatibility: what’s is missing? •Features with a better alternative in Azure OMS •Retired features •Features considered post-GA
  • 98. CLR considerations Managed Instance cannot access file shares and Windows folders Only CREATE ASSEMBLY FROM BINARY is supported CREATE ASSEMBLY FROM FILE is not supported ALTER ASSEMBLY can’t reference files
  • 99. SQL Server Agent Built into Managed Instance Azure SQL Database requires using on-premises SQL Server Agent, Azure Automation, Elastic Jobs, or PowerShell Always running Services cannot be stopped or restarted like they can with on-premises Option to auto-restart SQL Server if it stops unexpectedly is disabled Option to auto-restart SQL Server Agent if it stops unexpectedly is disabled Forwarding SQL Server events is disabled On-premises SQL Server Agent allows for forwarding events to another server but this is currently not an option for a Managed Instance Connection Alias local host server is predefined for a Managed Instance, whereas on-premises SQL Server Agent allows that to be configured if needed Creating jobs Creating jobs is as simple and easy as on-premises Jobs can be created using the UI or T-SQL Alert System Functions the same as on-premises for sending email alerts SQLCMD Cannot be called within a SQL Server Agent job Can be used to connect to a Managed Instance
  • 100. Service Broker within instances Service broker is on by default for all user databases Cross-instance service broker is not supported CREATE ROUTE does not work with ADDRESS other than LOCAL ALTER ROUTE does not work with ADDRESS other than LOCAL
  • 101. Fully supported in Managed Instance Functions the same as on-premises to set up and use Azure SQL Database does not have Database Mail support Database mail
  • 102. Replication support Supported Snapshot replication. Same functionality as on-premises Transactional replication Unsupported Peer-to-peer replication Merge replication Heterogeneous replication Oracle publisher For comparison, Azure SQL Database only supports being a transactional replication push subscriber Some restrictions when used with a Managed Instance Updatable subscriptions are not permitted Publisher and distributor must be in the same location If publisher and distributor are in a Managed Instance, Azure file share must be used to store data and schema from the publication Connections to the Distributor must use SQL authentication Additions to support Managed Instance New fields have been added in replication-related tables in msdb job_login, job_password, storage_connection_string SSMS replication wizard supports using a Managed Instance
  • 103.
  • 104. User DB file layout considerations Data file default initial size is 16MB with 16MB auto growth These can and should be adjusted for your workload File size limit is 8TB in General Purpose Log file default initial size is 8MB with 16MB auto growth This can and should be adjusted for your workload Additional data files/filegroups can be added Only using an ALTER DATABASE statement and the FILENAME clause is not permitted Paths and File Names are chosen for you Different from Azure SQL Database where additional files are not allowed Multiple log files are not supported (and should not be needed) A backup with multiple files/filegroups can be restored Each user database has a FILESTREAM filegroup for In-Memory OLTP checkpoint files Multiple log files are not supported (and should not be needed)
  • 105. TempDB data file considerations Tempdb Tuning Options Additional tempdb data files can be created if needed Well-known tempdb tuning ‘fixes’ are on by default Tempdb Resizing
  • 106. Backups are automatic Database backup schedule is the same as Azure SQL Database COPY_ONLY, URL-based backups can be used to perform manual full database backups Backup retention is 7 days by default
  • 107. Restore considerations Point-in-time restores are possible and must be performed manually using the Azure Portal Restoring automated backups from within SSMS is not allowed You can only restore using the Azure Portal COPY_ONLY, URL-based full backups can be restored using SSMS to a Managed Instance only Cannot be restored to on-premises as Managed Instance uses a higher build than on-premises instances Databases with multiple log files cannot be restored Secondary log files must be removed prior to backing up and restoring to a Managed Instance Can restore backups in a specific DB Compatibility Supports up to SQL 2005
  • 108. Migrate an on-premise SQL Server to Azure SQL Database Ike Ellis Solliance General Manager – Data + AI Practice Microsoft MVP
  • 109. Agenda • How Azure SQL DB Works • Deployment models • Purchasing options • Service tiers • Serverless compute • Hyperscale • T-SQL differences • High availability • Geo-replication • DEMO: Configuring geo-replication • Backup and restore • Retention • Accelerated Database Recovery • Security • BacPacs • Demo: Scripted migration using bacpacs
  • 110. How Azure SQL DB works • logical vs physical server
  • 111. Deployment models • Single database • Each database is isolated from others and is portable • Elastic pool
  • 112. Purchasing options • vCore • you choose the exact amount of compute resources that are always provisioned for your workload • DTU • bundled compute and storage packages balanced for common workloads • usually cheaper • only available with Azure SQL DB (not MI)
  • 113. Service tiers • General Purpose • Business Critical • Super-fast local SSDs • Better log throughput • 1 read-only replica built-in (can have 4 using geo-replication, but so can GP)
  • 114. Serverless • Automatically scales compute • Minimum & maximum vCores • Autopause delay • Good for: • Single databases with intermittent, unpredictable usage patterns interspersed with periods of inactivity • New single databases without usage history where compute sizing is difficult or not possible to estimate prior to deployment in SQL Database.
  • 115. Hyperscale • Great solution for data marts and analytics systems – good for loading because of low log contention • Great solution for high-load, large transactional systems
  • 117. High availability – General Purpose
  • 118. High availability – Business Critical
  • 120. Backup/restore • full backups every week • differential backups every 12-24 hours • transaction log backups every 5 to 10 minutes • PITR • Azure SQL Database retain sufficient backups to allow PITR within the last 7 days by default. • You can change backup retention period per each active database in the 1-35 day range. • You can configure full backup long-term retention (LTR) for up to 10 years in Azure Blob storage
  • 122. Accelerated Database Recovery in Azure SQL • Fast and consistent database recovery • long running transactions do not impact the overall recovery time • enabling fast and consistent database recovery irrespective of the number of active transactions in the system or their sizes. • Instantaneous transaction rollback • transaction rollback is instantaneous • Aggressive log truncation
  • 123.
  • 124. sLog • a secondary in-memory log stream that stores log records for non-versioned operations (such as metadata cache invalidation, lock acquisitions, and so on). The sLog is: • Low volume and in-memory • Persisted on disk by being serialized during the checkpoint process • Periodically truncated as transactions commit • Accelerates redo and undo by processing only the non-versioned operations • Enables aggressive transaction log truncation by preserving only the required log records •
  • 125. Security • Threat detection • TLS • TDE • Azure Key Vault • Always Encrypted • Dynamic Data Masking • Data discovery and classification
  • 126. Migrating to Azure SQL Database using bacpac • In practice, often includes multiple databases • Scripts are far easier to implement when there are a lot of databases
  • 127. Database compatibility considerations • Feature compatibility • https://docs.microsoft.com/en-us/azure/azure-sql/database/transact-sql-tsql-differences-sql-server#features-not-supported-in-sql-database • Behavior compatibility • While built on a common codebase, Azure SQL Database behaves differently from SQL Server in several important areas, i.e. high availability, backup, connectivity, security, resource governance, default concurrency model, etc. Most importantly, the application must implement robust retry logic to work reliably once migrated to the cloud
  • 128. Test the migration, test the applications in Azure SQL DB • Use the DMA Assessment • Build the project in SSDT • Uses the same DacFX framework and bacpac file as Azure SQL DB
  • 129. Migration size limits • Diskspace requirements holding the bacpac file • File has a hard time above 400GB • Maximum size of a block blob in ABS is 4TB
  • 130. Migration step detailed steps (most we’ve already covered) 1. Provision Azure resources 2. Declare the start of application downtime. 3. Export a bacpac of the source database. This bacpac contains all database scoped objects and data to be migrated. 4. Upload the bacpac to Azure Blob Storage 5. Start and monitor a bacpac import operation. 6. Once the import operation completes successfully, verify objects and data in the migrated database. 7. Grant database access. 8. Verify application functionality, as well as database management and monitoring functionality. 9. Declare the end of application downtime.
  • 131. Export the bacpac • SqlPackage /Action:Export /SourceServerName:SampleSQLServer.sample.net,1433 /SourceDatabaseName:SampleDatabase /TargetFile:"F:TempSampleDatabase.bacpac“ • All write access to the database should be stopped to keep the bacpac transactionally consistent
  • 132. Upload the bacpac • Needs a storage account • AzCopy is a good, scriptable method for doing this • AzCopy copy "D:Temptsqlv5.bacpac" "https://sqlbackupike.blob.core.windows.net/backups/?sv=2019-12- 12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2020-10-30T10:04:03Z&st=2020-09-28T02:04:03Z&sip=0.0.0.0- 255.255.255.255&spr=https&sig=iZW%2F6U7lxLvZO2MoRA8tdBshU2HOUwC2Fihyzhg%3D"
  • 133. Import the bacpac • Can do this in the portal • Or script it: • sqlpackage.exe /a:import /tcs:"Data Source=<serverName>.database.windows.net;Initial Catalog=<migratedDatabase>;User Id=<userId>;Password=<password>" /sf:AdventureWorks2008R2.bacpac /p:DatabaseEdition=Premium /p:DatabaseServiceObjective=P6
  • 134. Verify migrated data SELECT s.name AS [schema_name], o.name AS [object_name], o.type_desc AS [object_type_desc] FROM sys.objects AS o INNER JOIN sys.schemas AS s ON o.schema_id = s.schema_id WHERE s.name <> 'sys' AND o.is_ms_shipped = 0 AND o.type NOT IN ('IT','S') ;
  • 135. Verify with row count SELECT s.name AS [schema_name], t.name AS table_name, SUM(p.rows) AS row_count FROM sys.partitions AS p INNER JOIN sys.indexes AS i ON p.object_id = i.object_id AND p.index_id = i.index_id INNER JOIN sys.tables AS t ON p.object_id = t.object_id INNER JOIN sys.schemas AS s ON t.schema_id = s.schema_id WHERE s.name <> 'sys' AND t.is_ms_shipped = 0 AND t.type NOT IN ('IT','S') AND i.type_desc IN ('HEAP','CLUSTERED','CLUSTERED COLUMNSTORE') GROUP BY s.name, t.name;
  • 136. Post migration tasks after migrating databases to Azure offerings Ike Ellis Solliance General Manager – Data + AI Practic Microsoft MVP
  • 137. You did it! You migrated! Now what?
  • 138. Governance • Governance is commonly defined as a set of policies, roles, responsibilities and processes that control a business system or systems and how the various teams work together to achieve set goals
  • 139. Governance: Security breaches • How will you handle them? • What is the response? • Who is responsible? • https://docs.microsoft.com/azure/governance/azure-management
  • 140. Governance • Will span the lifecycle of your Azure resources through the following major stages: • Migrate : On-premises workload migrations • Secure : Security and threat management • Protect : Backup and recovery • Monitor : Apps, infrastructure, network, logs and diagnostics • Configure : Update management, automation and scripting • Govern : Policy and cost management
  • 141. Management • Many management tasks can be performed, and automated, using the Azure VM Portal • Register the SQL Server virtual machine in Azure with the SQL VM resource provider • Creates the SQL Virtual Machine resource within the Azure subscription • Registering the SQL Server on an Azure VM as a SQL Server VM resource may help streamline licensing and reduce costs.
  • 142. SQL Server IAAS Agent Extension • The SQL Server and Azure team developed a helpful provider and agent to support common administrative tasks • The SQL Server IaaS Agent Extension runs on Azure virtual machines to automate administration tasks. • Common tasks include: • Automated backup • Automated patching • Azure Key Vault integration
  • 143. Cost Optimization • Azure Advisor • a must-use tool for your Azure Subscriptions. It can help identify workloads that can be scaled down • Azure Cost Management and Billing • Tracks spending in your Azure subscriptions and resource groups • Azure Budgets • Proactively manage and monitor costs and get alerts on excessive spending
  • 144. Rebuild all your indexes • You may have new disks, new optimizer, new hardware • Or a new product, like Azure SQL Database
  • 145. Statistics • Rebuild these too • Particularly on stats that aren’t associated with indexing
  • 146. BCDR Plan • Business Continuity and Disaster Recovery • Your plan has now changed that you are in Azure • You need an updated plan that accounts for a new RTO and RPO • Test the HADR plan • On VMs, do you need to build an AG? • If so, are you using Availability Sets or Availability Zones?
  • 147. Backup/Restore strategy • You need to document how you backup and restore • When you restore • What’s your Long Term Retention (LTR) policy • How you are staying in compliance with it • How you are testing it • Test it regularly
  • 148. Extra training • If you want to gain valuable experience working with tools like Azure Site Recovery and Azure Backup, reference the Microsoft Cloud Workshops and the Business Continuity and Disaster Recovery workshop.
  • 149. Security - Azure Security Center • Vulnerability assessment is an easy to configure service that can discover, track, and help you remediate potential database vulnerabilities • Advanced Threat Protection detects anomalous activities indicating unusual and potentially harmful attempts to access or exploit your SQL server. - $15/month • Advanced Data Security for SQL Servers
  • 150. Virtual Machine - Antimalware • You should install antimalware on all your virtual machines. • Azure Security Center can also be used to deploy and integrated security solutions automatically to your Virtual Machine workloads • Identify VMs that have public endpoints
  • 151. Security: Just in time • Remote Desktop is one of the biggest entry points for attackers. • Brute force tools can continually attempt to login to the servers until a successful password has been found. • Just in time (JIT) access can be enabled by default to deny remote access until an administrator actually needs access • Just in time access must be requested and approved, all of which is audited in the Azure control plane and queryable via Kusto queries in Log Analytics. • Once access has been granted, it will be limited to the administrator's IP address and for a set amount of time.
  • 152. Windows Firewall • Enable it • Use it • Love it
  • 153. Auditing • Logs can be stored • Routed to Azure Log Analytics • Queried using Kusto
  • 154. Snapshot the VM • If you are going to make any major changes to your virtual machine that may have ramifications of taking down your data workloads, it is important that you utilize virtual machine snapshots in case you want to roll back those changes • https://docs.microsoft.com/azure/virtual-machines/windows/snapshot-copy-managed-disk
  • 155. Consider Data Virtualization Big SQL Database Move cold tables Move Blobs Parquet Files T-SQL Create Table Statement SELECT * FROM <table in SQL> JOIN <table in Azure Blob Storage> ON…… Smaller/faster /cheaper SQL Database Ready to move to PAAS
  • 156. Platform Modernization: SSAS • Azure Analysis Services – PAAS Service • Only supports tabular • Doesn’t support multi-dimensional • Not all data sources available in SSAS are available • Not all DAX Formulas are supported • Some limitations with DirectQuery in tabular models • Migration: You can backup and restore! • Or re-deploy and reprocess
  • 157. Platform Modernization SSAS • Consider it the networking setup between the PaaS Service and the on-premises data sources • Processing might take longer • ExpressRoute is still key!
  • 158. Platform Modernization: SSIS • Move to Azure Data Factory • SSIS can either be converted, or run inside of ADF • Data Migration Assistant can help! • We saw it on a SQL Server database • It will work on SSIS, too • Will flag any migration blockers and highlight recommendations around deprecated features and other potential concerns that may not be blockers
  • 159. Platform Modernization: SSIS • Migrate SSIS catalog (SSISDB) • Can move to Azure SQL Database MI
  • 160. Platform Modernization: SSRS • You love it, I love it, now what do we do with it? • Keep it in a VM • Deploy new reports to Power BI • RDL Migration Tool: https://github.com/microsoft/RdlMigration
  • 161. Platform Modernization: SSRS • The tool will only migrate the RDL files, the following are not supported: • Shared data sources • Shared datasets • Resources, like image files • KPIs (SSRS 2016, or later—Enterprise Edition only) • Mobile reports (SSRS 2016, or later—Enterprise Edition only) • Report models (deprecated) • Report parts (deprecated)
  • 162. but how do we get from here: huge sql database app 1 app 2 app 3 excel ssrs reports ssis etl custom app data dump to vendor sql agent job that closes the month somebody hand made a lookup table for one report linked server for accounting process stored procs on a different server email notification for alerting power bi
  • 163. step 3: now that you’ve de-coupled the database using mi, let’s see if you can move to paas much smaller sql database app 1 cosmosdb app 2 much smaller sql database app 3 remember this is the goal how do we get there?
  • 164. the main problems with legacy applications • too many dependencies • bleeding layers • code that is unreadable • code that no one wants to change • repeatedly introducing the same bugs
  • 166. stinting / seam testing a seam is a place where you can alter behavior in your program without editing in that place. we split the method into two methods without changing functionality this allows us to cover one or both of the methods in test once covered in tests, we can begin to change functionality
  • 167. testing strategies • imagine we’re pulling a puzzle apart, but leaving the pieces where they are • soon we have separate pieces, but now we can look at them individually • move the individual piece to a paas offering • can it be deployed separately
  • 168. only convert that subset to paas good candidates are: • archival • authentication • notification services
  • 169. concentrated re-write only rewrite a small target that you can rip out of the legacy app
  • 170. let the developers choose their data store guide them towards the products you know develop your expertise
  • 171. remember our main rules only the application touches the database all integration is done through an event hub the bi teams can consume the hub too…..they don’t need to touch the db
  • 172. what’s a good boundary? think in terms of deployment what features are likely to be tested and deployed together keep the testing surface area small-enough but not too small deploy the database with the code version both of them think in terms of remediation. how are conflicts resolved?
  • 173. the death of the dba, you are now an azure data architect • roles and responsibilities • day to day life • rise of your career
  • 174. things you no longer need to do • sweat about disk space • worry about patch level and security patches • create an upgrade plan • care about what the ops guys are doing • wonder if you’re going to get priority on the san • purchase hardware • argue about capital expenses
  • 175. things you absolutely have to do • follow every blog and twitter account for any product you’re responsible for • Life moves pretty fast. If you don't stop and look around once in a while, you could miss it. • learn as much as you can about a wide variety of products • constantly interact with the developers. it’s your day to watch them. • de-couple everything from your schema so you can normalize or de-normalize as necessary • consider partitioning • learn scripting because the number of database… • well, they’re about to explode
  • 176. day to day life • create scripts for all manual activities • learn terraforming • learn scripting and automation • create framework to easily deploy dev/test environments • learn docker/Kubernetes • learn python probably • watch your data model • watch your performance • predict scale-up/scale-down • watch your bill • watch those developers
  • 177. rise of your career • as you embrace the cloud, you will find lots of future opportunities • no longer is knowledge about only one database product acceptable • learn no-sql • learn what mpp and analytics teams are doing • learn about data pipelining • double your income
  • 178. Ike Ellis, MVP General Manager – Data & AI Practice Solliance @ike_ellis www.ikeellis.com youtube.com/IkeEllisOnTheMic ike@solliance.net • Founder of San Diego Power BI and PowerApps UserGroup • Founder of the San Diego Software Architecture Group • MVP since 2011 • Author of Developing Azure Solutions, Power BI MVP Book • Speaker at PASS Summit, SQLBits, DevIntersections, TechEd, Craft, Microsoft Azure & AI Conference

Editor's Notes

  1. Rehost & refactor are the by far the simplest way to migrate and can be seamless. Obviously, Rearchitecting and rebuilding can take considerable effort and time.
  2. We’ve recently announced support for reserved capacity. Customers achieve the lowest cost of ownership by combining reservation pricing (either 1- or 3-years) with the Azure Hybrid Benefit. Customers also have the option to combine reservation pricing with license-included prices. Reservations help customers manage costs across predictable and variable workloads and improve budgeting and forecasting with a single upfront payment, making it easy to calculate investments. While Azure Reserved Instances require making upfront commitments on capacity, they also provide flexibility should your business needs change. You can easily exchange or cancel reserved instances at any time. Reserved Instances can be assigned at the enrollment or subscription level, so you can manage reserved instance usage at an organizational or individual department level. Assignments are easy to change post-purchase also.
  3. Create as s
  4. Who is Managed Instance for? Customers looking to migrate a large number of apps from on-premise or IaaS, self-built or ISV provided, with as low migration effort as possible & cost being a crucial factor
  5. SQL Database Managed Instance provides complete workload isolation of your workloads through native VNET support. We use virtual data clusters to describe the degree of isolation that customer workloads will experience with SQL Database Managed Instance. During service provisioning (on Azure Portal or through REST API), you can choose the virtual network (VNET) and the subnet to achieve full networking isolation for your Managed Instances. Once created, instances in the VNET can be reached using Azure networking mechanisms (VPN and Express Route gateways). To two levels of isolation are provided: Cluster (tenant ring) level: Managed Instances for a tenant are fully isolated from other tenants. No connectivity or resource sharing is possible between different tenants. Networking level: joining instances to a subnet in a VNET and restricting access to private IP addresses only provides full isolation from public Internet.
  6. New deployment option enabling friction-free migration of SQL Server workloads to a fully-managed service
  7. Key points: Enable full isolation from other tenants without resource sharing Promote secure communication over private IP addresses with native VNET integration Enable your on-premise identities on cloud instances, through integration with Azure Active Directory and AD Connect To two levels of isolation are provided: Cluster (tenant ring) level: Managed Instances for a tenant are fully isolated from other tenants. No connectivity or resource sharing is possible between different tenants. Networking level: joining instances to a subnet in a VNET and restricting access to private IP addresses only provides full isolation from public Internet. SQL Database Managed Instance provides complete workload isolation of your workloads through native VNET support. We use virtual data clusters to describe the degree of isolation that customer workloads will experience with SQL Database Managed Instance. During service provisioning (on Azure Portal or through REST API), you can choose the virtual network (VNET) and the subnet to achieve full networking isolation for your Managed Instances. Once created, instances in the VNET can be reached using Azure networking mechanisms (VPN and Express Route gateways).
  8. Prices on Sept 14: SQL MI GP (8 vCore): 2.02 USD / hour SQL MI BC (8 vCore): 5.44 USD / hour (equivalent GA price) AWS RDS SQL SE single AZ (8 vCPU): 3.02 USD / hour AWS RDS SQL EE multi AZ (8 vCPU): 10.00 USD / hour
  9. Be empty: The subnet must not contain any other cloud service associated to it, and it must not be Gateway subnet. You won’t be able to create Managed Instance in subnet that contains resources other than managed instance or add other resources inside the subnet later. Have specific route table: The subnet must have a User Route Table (UDR) with 0.0.0.0/0 Next Hop Internet as the only route assigned to it. Optional custom DNS: If custom DNS is specified on the VNet, Azure's recursive resolvers IP address (such as 168.63.129.16) must be added to the list. No Service endpoint: The subnet must not have a Service endpoint (Storage or Sql) associated to it. Make sure that Service Endpoints option is Disabled when creating VNet. Sufficient IP addresses: The subnet must have minimum of 16 IP addresses. For more information. By design, a Managed Instance needs a minimum of 16 IP addresses in a subnet and may use up to 256 IP addresses. As a result, you can use subnet masks /28 to /24 when defining your subnet IP ranges. Azure uses five IP addresses in the subnet for its own needs Each General Purpose instance needs two addresses Each Business Critical instance needs four addresses
  10. A Managed Instance must be deployed in an Azure Virtual Network Allows for connecting directly from an on-premises network Allows for connecting linked servers or other on-premises data stores Allows for connecting to additional Azure resources Plan your deployment Managed Instance requires a minimum of 16 IP addresses in a subnet and may use up to 256 IP addresses If deploying multiple Managed Instances inside the subnet, you need to optimize the subnet size The default values create a subnet that takes all the VNet address space, allowing for only Managed Instance inside the virtual network Routes Effective routes on the Managed Instance subnet are not supported Routes can be user-defined (UDR) or Border Gateway Protocol (BGP) routes propagated to network interfaces through ExpressRoute or site-to-site VPN connections For BGP routes, create a 0.0.0.0/0 Next Hop Internet route and apply it to the Managed Instance subnet Network Security Groups (NSG) NSGs on the Managed Instance subnet are not supported
  11. High Availability is handled behind the scenes automatically For General Purpose there is a four-node Availability Group The secondary is non-readable For Business Critical there is a three-node Availability Group One secondary is readable, and the other is non-readable (Application Intent = ReadOnly) An on-prem SQL Server Availability Group cannot be extended to a Managed Instance
  12. MI is always on latest and greatest SQL engine version Certify your code for database compatibility level not for a version Take advantage of new features (Temporal, JSON, Graph Database, etc.) Use rich T-SQL surface area, check out documentation page Your code can be SQL deployment model aware if necessary SERVERPROPERTY (‘EngineEdition’) = 8 uniquely identifies MI Current limitations (will be removed later this year) Time is UTC . Use AT TIME ZONE to add local time zone experience Instance collation is fixed (affects tempdb and system databases)
  13. Microsoft Database Compatibility Level Protection Full Functional protection once assessment tool runs clean. Maintaining backward compatibility is very important to SQL Server team. Query Plan shape protection. Overall process Use Database Migration Assistant (DMA) and Database Experimentation Assistant (DEA) for assessment. Migrate database and keep/set source Database Compatibility Level on target. Perform minimal testing or as determined by your organization. Contact Microsoft – Explore jointly on how to use Database Compatibility based certification.
  14. Features with a better alternative in Azure Always-On Availability Groups: local HA, active geo-replication Windows Authentication: Azure Active Directory is the alternative. Management Data Warehouse : OMS integration is the alternative. Retired features Database Mirroring: built-in HA / geo-replication Extended stored procedures: customers should use CLR Features considered post-GA Filestream, Filetable Cross-instance distributed transactions (MS DTC) Stretch Database PolyBase