HA/DR options with SQL Server in Azure and hybrid

1,536 views

Published on

What are all the high availability (HA) and disaster recovery (DR) options for SQL Server in a Azure VM (IaaS)? Which of these options can be used in a hybrid combination (Azure VM and on-prem)? I will cover features such as AlwaysOn AG, Failover cluster, Azure SQL Data Sync, Log Shipping, SQL Server data files in Azure, Mirroring, Azure Site Recovery, and Azure Backup.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,536
On SlideShare
0
From Embeds
0
Number of Embeds
435
Actions
Shares
0
Downloads
154
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • HA/DR options with SQL Server in Azure and hybrid
    What are all the high availability (HA) and disaster recovery (DR) options for SQL Server in a Azure VM (IaaS)?  Which of these options can be used in a hybrid combination (Azure VM and on-prem)?  I will cover features such as AlwaysOn AG, Failover cluster, Azure SQL Data Sync, Log Shipping, SQL Server data files in Azure, Mirroring, Azure Site Recovery, and Azure Backup.
  • Fluff, but point is I bring real work experience to the session
  • https://blogs.msdn.microsoft.com/mast/2013/12/06/understanding-the-temporary-drive-on-windows-azure-virtual-machines/

    SSD/HDD storage included in A-series, D-series, and Dv2-series VMs is local temporary storage.

    DS-series, G-series, GS-series SSD's have less local temporary storage due to storage used for caching purposes to ensure predictable levels of performance associated with premium storage.
    DS-series and GS-series support premium storage disks, which means you can attach SSD's to the VM (the other series support only standard storage disks).
    The pricing and billing meters for the DS sizes are the same as D-series and the GS sizes are the same as G-series.

    When you create an Azure virtual machine, it has a disk for the operating system mapped to drive C (size is 127GB) that is on Blob storage and a local temporary disk mapped to drive D. You can choose standard disk type or premium (if DS-series or GS-series) for your local temporary disk - the size of which is based on the series you choose (i.e. A0 is 20GB). You can also attach new disks - specify standard or premium, for standard: specify size (1GB-1023GB), for premium: specify P10, P20, or P30. the disks are .vhd files that reside in an Azure storage account
  • http://www.jamesserra.com/archive/2015/11/redundancy-options-in-azure-blob-storage/

    Geo-replication in Azure disks does not support the data file and log file of the same database to be stored on separate disks. GRS replicates changes on each disk independently and asynchronously. This mechanism guarantees the write order within a single disk on the geo-replicated copy, but not across geo-replicated copies of multiple disks. If you configure a database to store its data file and its log file on separate disks, the recovered disks after a disaster may contain a more up-to-date copy of the data file than the log file, which breaks the write-ahead log in SQL Server and the ACID properties of transactions. If you do not have the option to disable geo-replication on the storage account, you should keep all data and log files for a given database on the same disk. If you must use more than one disk due to the size of the database, you need to deploy one of the disaster recovery solutions listed above to ensure data redundancy. https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-sql-high-availability-dr/
  • https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-sql-dr/

    It is up to you to ensure that your database system possesses the HADR capabilities that the service-level agreement (SLA) requires. The fact that Azure provides high availability mechanisms, such as service healing for cloud services and failure recovery detection for the Virtual Machines (https://azure.microsoft.com/en-us/blog/service-healing-auto-recovery-of-virtual-machines), does not itself guarantee you can meet the desired SLA. These mechanisms protect the high availability of the VMs but not the high availability of SQL Server running inside the VMs. It is possible for the SQL Server instance to fail while the VM is online and healthy. Moreover, even the high availability mechanisms provided by Azure allow for downtime of the VMs due to events such as recovery from software or hardware failures and operating system upgrades.
  • https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-sql-server-alwayson-availability-groups-gui-arm/

    https://blogs.msdn.microsoft.com/igorpag/2013/09/02/sql-server-2012-alwayson-availability-group-and-listener-in-azure-vms-notes-details-and-recommendations/

    https://blogs.msdn.microsoft.com/igorpag/2014/07/03/deep-dive-sql-server-alwayson-availability-groups-and-cross-region-virtual-networks-in-azure/

    https://blogs.technet.microsoft.com/dataplatforminsider/2014/06/19/sql-server-alwayson-availability-groups-supported-between-microsoft-azure-regions/
  • https://blogs.technet.microsoft.com/dataplatforminsider/2014/06/19/sql-server-alwayson-availability-groups-supported-between-microsoft-azure-regions/

    https://blogs.msdn.microsoft.com/igorpag/2014/12/22/sql-server-2014-high-availability-and-multi-datacenter-disaster-recovery-with-multiple-azure-ilbs/
  • https://azure.microsoft.com/en-us/blog/high-availability-for-a-file-share-using-wsfc-ilb-and-3rd-party-software-sios-datakeeper/

    Replaces SQL Server failover cluster

    Requires shared storage
  • https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-sql-server-backup-and-restore/

    https://blogs.technet.microsoft.com/dataplatforminsider/2016/04/19/sql-server-2016-cloud-backup-and-restore-enhancements/
  • https://msdn.microsoft.com/en-us/library/dn385720.aspx
  • https://azure.microsoft.com/en-us/documentation/articles/site-recovery-overview/

  • https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-manage-availability/

    https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-configure-availability/

    In ARM:
    You can’t change the VM’s Availability Set once the VM is created
    You can’t add an Azure VM to an Availability Set once the VM is created
    You can’t remove a VM from an Availability Set
    Instead use Powershell: https://buildwindows.wordpress.com/2016/02/25/add-or-change-an-arm-virtual-machines-availability-set/
  • https://azure.microsoft.com/en-us/documentation/articles/sql-database-get-started-sql-data-sync/

    https://www.mssqltips.com/sqlservertip/3062/understanding-sql-data-sync-for-sql-server/

    https://msdn.microsoft.com/en-us/library/hh868047.aspx

    Compare SQL Data Sync to Active Geo-Replication
  • When it comes to hybrid solution we are introducing new industry first concepts of stretch tables into Azure for operational databases. So think about it

    We are making migrating SQL Server from on-prem to Azure VMs much easier. Currently when you move SQL Server we only migrated the schema and data. With SQL v-next you will able to migrate systems objects, SQL Settings so that migration is literally point and click type of migration. We will provide a wizard drive experience that will provide gallery image, vm size recommendations.
  • Notes
    Now let’s talk about some of these innovative hybrid scenarios that can compliment your on-premises SQL Server investments. Stretch database, is one of these unique hybrid scenarios that only Microsoft provides and can be extremely valuable in your data strategy.
    We know your OLTP databases are growing rapidly and you need to think about how to cost effectively manage your data. More importantly, you need to think about what you’re doing with historical data and whether you want it to go offline onto tape. When it goes to tape, it’s not queryable anymore. What if you wanted the historical data at your fingertips but didn’t want to have it reside on premium storage with your hot data because it’s too costly.
    We can solve that problem with this stretch database hybrid scenario. Without modifying your app, you can stretch your database to Azure. You don’t even have to do any of the heavy lifting to make this work. You just have to set the policy you want to apply on the historical data and run the query as you normally would. SQL Server determines if the table has been stretched and retrieves the data from Azure. The other thing to note is that this technology does not impact the performance of writes on the same table being stretched as part of the engineering design point for this solution. A question you might ask is I am stretching historical customer data which has sensitive data, what about data security. The best part is that stretch database works with our new Always Encrypted technology that protects the columns you desire at rest and in motion, even in the memory buffer pool, so sensitive customer information is secure.
    Stretch database offers a great value prop in terms of saving you money and providing easy access to historical customer data so you can make improve customer experiences without requiring application modifications.
    This is just one example of how cloud-first innovation allows us to deliver new hybrid scenarios that our competitor simply can’t deliver or haven’t thought of.
     
  • Source: https://msdn.microsoft.com/en-us/library/dn935011(v=sql.130).aspx

    Stretch Database lets you archive your historical data transparently and securely. In SQL Server 2016 Community Technology Preview 2 (CTP2), Stretch Database stores your historical data in the Microsoft Azure cloud. After you enable Stretch Database, it silently migrates your historical data to an Azure SQL Database.
    You don't have to change existing queries and client apps. You continue to have seamless access to both local and remote data.
    Your local queries and database operations against current data typically run faster.
    You typically enjoy reduced cost and complexity.


  • Source: https://msdn.microsoft.com/en-us/library/mt169378(v=sql.130).aspx

    Concepts and architecture for Stretch Database
    Terms
    Local database. The on-premises SQL Server 2016 Community Technology Preview 2 (CTP2) database.
    Remote endpoint. The location in Microsoft Azure that contains the database’s remote data. In SQL Server 2016 Community Technology Preview 2 (CTP2), this is an Azure SQL Database server. This is subject to change in the future.
    Local data. Data in a database with Stretch Database enabled that will not be moved to Azure based on the Stretch Database configuration of the tables in the database.
    Eligible data. Data in a database with Stretch Database enabled that has not yet been moved, but will be moved to Azure based on the Stretch Database configuration of the tables in the database.
    Remote data. Data in a database with Stretch Database enabled that has already been moved to Azure.

    Architecture
    Stretch Database leverages the resources in Microsoft Azure to offload archival data storage and query processing.

    When you enable Stretch Database on a database, it creates a secure linked server definition in the on-premises SQL Server. This linked server definition has the remote endpoint as the target. When you enable Stretch Database on a table in the database, it provisions remote resources and begins to migrate eligible data, if migration is enabled.

    Queries against tables with Stretch Database enabled automatically run against both the local database and the remote endpoint. Stretch Database leverages processing power in Azure to run queries against remote data by rewriting the query. You can see this rewriting as a "remote query" operator in the new query plan.
  • Source: https://msdn.microsoft.com/en-us/library/mt163698(v=sql.130).aspx

  • HA/DR options with SQL Server in Azure and hybrid

    1. 1. James Serra HA/DR options with SQL Server in Azure and hybrid
    2. 2. About Me  Microsoft, Big Data Evangelist  In IT for 30 years, worked on many BI and DW projects  Worked as desktop/web/database developer, DBA, BI and DW architect and developer, MDM architect, PDW/APS developer  Been perm employee, contractor, consultant, business owner  Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data World conference  Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions  Blog at JamesSerra.com  Former SQL Server MVP  Author of book “Reporting with Microsoft SQL Server 2012”
    3. 3. Agenda  VM storage  AlwaysOn AG  AlwaysOn FCI  Database Mirroring  Log Shipping  Backup to Azure  SQL Server data files in Azure  Azure Site Recovery  Azure SQL Data Sync
    4. 4. Virtual Machine storage architecture C: OS disk (127 GB) E:, F:, etc. Data disks (1 TB) Attach SSD/HDD up to 1TB. These are .vhd files D: Temporary disk (Contents can be lost) SSD/HDD and size depends on VM chosen Disk Cache
    5. 5. Azure Default Blob Storage  Azure Storage Page Blobs, 3 copies  Storage high durability built-in (like have RAID)  VHD disks, up to 1 TB per disk (64 TB total)
    6. 6. Geo-storage replication  3 copies locally, another 3 copies in different region  Disable for SQL Server VM disk (consistent write order across multiple disks is not guaranteed). Instead use DR techniques in this deck Defend against regional disasters Geo replication
    7. 7. HA/DR deployment architectures Azure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. Cross-region solution protects against complete site outage. Hybrid Some availability replicas running in Azure VMs and other replicas running on- premises for cross- site disaster recovery. HA only, not DR FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution. FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute. Azure Only Principal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA. Hybrid One partner running in an Azure VM and the other running on- premises for cross- site disaster recovery using server certificates. For DR only / Hybrid only One server running in an Azure VM and the other running on- premises for cross- site disaster recovery. Log shipping depends on Windows file sharing, so a VPN connection between the Azure virtual network and the on- premises network is required. Requires AD deployment on DR site. On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery. SQL 2016: Backup to Azure with file snapshots Simpler BCDR story Site Recovery makes it easy to handle replication, failover and recovery for your on-premises workloads and applications (not data!). Flexible replication You can replicate on- premises servers, Hyper-V virtual machines, and VMware virtual machines. Eliminate the need for secondary Native support for SQL Server data files stored as Azure blobs
    8. 8. RPO/RTO RTO – Recover Time Objective. How much time after a failure until we have to be up and running again? RPO – Recover Point Objective. How much data can we lose? • HA – High Availability • RTO: seconds to minutes • RPO: Zero to seconds • Automatic failover • Well tested (maybe with each patch or release) • DR – Disaster Recovery • RTO: minutes to hours • RPO: seconds to minutes • Manual failover into prepared environment • Tested from time to time How long does it take to fail over: • Backup-Restore: Hours • Log Shipping: Minutes • AlwaysOn FCI: Seconds to minutes • AlwaysOn AG/Mirroring: Seconds
    9. 9. AlwaysOn Availability Groups Azure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. Cross-region solution protects against complete site outage. Hybrid Some availability replicas running in Azure VMs and other replicas running on- premises for cross- site disaster recovery. Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. This cross-region solution protects against complete site outage. Within a region, all replicas should be within the same cloud service and the same VNet. Because each region will have a separate VNet, these solutions require VNet to VNet connectivity. For more information, see Configure a Site-to-Site VPN in the Azure classic portal. All availability replicas running in Azure VMs for high availability within the same region. You need to configure a domain controller VM, because Windows Server Failover Clustering (WSFC) requires an Active Directory domain. For more information, see Configure AlwaysOn Availability Groups in Azure (GUI).
    10. 10. AlwaysOn Availability Groups (Hybrid) Azure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. Cross-region solution protects against complete site outage. Hybrid Some availability replicas running in Azure VMs and other replicas running on- premises for cross- site disaster recovery. Some availability replicas running in Azure VMs and other replicas running on-premises for cross-site disaster recovery. The production site can be either on-premises or in an Azure datacenter. Because all availability replicas must be in the same WSFC cluster, the WSFC cluster must span both networks (a multi- subnet WSFC cluster). This configuration requires a VPN connection between Azure and the on-premises network. For successful disaster recovery of your databases, you should also install a replica domain controller at the disaster recovery site. It is possible to use the Add Replica Wizard in SSMS to add an Azure replica to an existing AlwaysOn Availability Group. For more information, see Tutorial: Extend your AlwaysOn Availability Group to Azure.
    11. 11. AlwaysOn between Azure Regions • Configure AlwaysOn between VMs in different geographic regions (asynchronous) • Over secure tunnel • Manual Failover (~15 seconds) in case of a regional failure • Test it at any time • Use closest secondary for read workloads • Region 1: AG used instead of FCI (synchronous)
    12. 12. AlwaysOn Failover Cluster Instances (FCI) An FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute. For example, NetApp Private Storage (NPS) exposes an iSCSI target via ExpressRoute with Equinix to Azure VMs. For third-party shared storage and data replication solutions, you should contact the vendor for any issues related to accessing data on failover. Note that using FCI on top of Azure File storage is not supported yet, because this solution does not utilize Premium Storage. We are working to support this soon. HA only, not DR FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution. FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute. You can use FCI to host an availability replica for an availability group FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.
    13. 13. AlwaysOn FCI vs AlwaysOn AG
    14. 14. Database Mirroring Azure Only Principal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA. Hybrid One partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates. Principal and mirror and servers running in different datacenters for disaster recovery. You must deploy using server certificates because an Active Directory domain cannot span multiple datacenters. Principal, mirror, and witness servers all running in the same Azure datacenter for high availability. You can deploy using a domain controller. You can also deploy the same database mirroring configuration without a domain controller by using server certificates instead.
    15. 15. Database Mirroring vs AlwaysOn AG
    16. 16. Database Mirroring (Hybrid) Azure Only Principal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA. Hybrid One partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates. One partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates. Partners do not need to be in the same Active Directory domain, and no VPN connection is required. Another database mirroring sceanario involves one partner running in an Azure VM and the other running on-premises in the same Active Directory domain for cross-site disaster recovery. A VPN connection between the Azure virtual network and the on-premises network is required. For successful disaster recovery of your databases, you should also install a replica domain controller at the disaster recovery site.
    17. 17. Log Shipping (Hybrid) For DR only / Hybrid only One server running in an Azure VM and the other running on- premises for cross- site disaster recovery. Log shipping depends on Windows file sharing, so a VPN connection between the Azure virtual network and the on- premises network is required. Requires AD deployment on DR site.
    18. 18. Block blobs Reduced storage costs Significantly improved restore performance More granular control over Azure Storage Azure Storage snapshot backup Fastest method for creating backups and running restores Support of SQL Server database files on Azure Blob Storage Backup to Azure Managed backup On-prem to Azure Granular control of the backup schedule Local staging for faster recovery and greater network resiliency System database support Simple recovery mode support On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery. SQL 2016: Backup to Azure with file snapshots Production databases backed up directly to blob storage in a different datacenter for disaster recovery On-premises production databases backed up directly to Azure blob storage for disaster recovery.
    19. 19. Backup to Azure with file snapshots (SQL Server 2016) BACKUP DATABASE database TO URL = N'https://<account>.blob.core.windows.net/<container>/<backupfileblob1>‘ WITH FILE_SNAPSHOT Instance Azure Storage MDF Database MDF LDF LDF BAK Hybrid solutions
    20. 20. SQL Server data files in Azure (Hybrid) Native support for SQL Server data files stored as Azure blobs - Easy and fast migration benefits - Cost and limitless storage benefits - High availability and disaster recovery benefits - Security benefits - Snapshot backup
    21. 21. Azure Site Recovery (Hybrid) Simpler BCDR story Site Recovery makes it easy to handle replication, failover and recovery for your on-premises workloads and applications. Flexible replication You can replicate on- premises servers, Hyper-V virtual machines, and VMware virtual machines. Eliminate the need for secondary SQL Server on-prem DR example: Standalone SQL Server instance residing on-premises and replicating to an Azure Storage account by using Azure Site Recovery. The replication targets are page blobs containing the vhd files of Azure IaaS virtual machines hosting SQL Server instances that are brought online during failover.
    22. 22. Azure SQL Data Sync (preview) SQL Azure Data Sync is a Microsoft Windows Azure web service that provides data synchronization capabilities for SQL databases. SQL Azure Data Sync allows data to be synchronized between on-premises SQL Server databases and Azure SQL databases; in addition, it can also keep multiple Azure SQL databases in sync. SQL Data Sync targets the reference data replication scenario. Its key capabilities are:  Sync between SQL Server (2005 SP2 and later) and Azure SQL databases, or between Azure SQL databases  One-way and bi-directional sync  One-to-one and hub-spoke  Table filter and column filter  Scheduled and on-demand  Eventual consistency Active Geo-Replication, in contrast, targets GeoDR scenario for Azure SQL Database by replicating the database to another region. It only supports one- way replication (secondaries are read-only), replication is at database granularity and no database or column/row filter support, and it is only available for Premium service tier.
    23. 23. Resources SQL Server in VM best practices: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-sql- server-performance-best-practices/ https://azure.microsoft.com/en-us/documentation/articles/azure-subscription-service- limits/#virtual-machines-limits https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-size-specs/ https://azure.microsoft.com/en-us/pricing/details/virtual-machines/ Disaster Recovery and High Availability for Azure Applications: https://msdn.microsoft.com/en- us/library/azure/dn251004.aspx
    24. 24. © 2016 Microsoft Corporation. All rights reserved.

    ×