Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Best Practices running SQL Server on AWS


Published on

In this session, we will introduce how to deploy your SQL Server to AWS by RDS and EC2.

Published in: Technology
  • Can You Use Any Charger With Any Cell Phone, Laptop, Camera, or Tablet? ◆◆◆
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❤❤❤ ❤❤❤
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ♥♥♥ ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website!
    Are you sure you want to  Yes  No
    Your message goes here

Best Practices running SQL Server on AWS

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. John Chang ( 張書源 ) Technology Evangelist, AWS March 2017 SQL Server 在 AWS 的 最佳實踐
  2. 2. What to Expect from the Session • Microsoft SQL Server deployment options on AWS • Understanding licensing options • Best practices: • SQL Server on Amazon EC2 • Amazon RDS for SQL Server
  3. 3. AWS 現況 約 130 億美元 (過去12 個月,截止到2016 Q3) 55%成長 (2015 Q3 v.s. 2016 Q3) 數百萬 每月活躍用戶
  4. 4. AWS 全球基礎設施 16 區域 42 可用區域 新地理區域 巴黎 寧夏
  5. 5. AWS Global Infrastructure Regions Geographic locations Consists of at least two Availability Zones (AZs) Availability Zones Clusters of data centers Isolated from failures in other Availability Zones
  6. 6. Availability Zones (AZs) At least 2 AZs per region. Examples: • US East (N. Virginia) • us-east-1a • us-east-1b • us-east-1c • us-east-1d • us-east-1e • Asia Pacific (Tokyo) • ap-northeast-1a • ap-northeast-1b • ap-northeast-1c Note: Conceptual drawing only. The number of Availability Zones (AZ) may vary. US East (VA) AZ - A AZ - B AZ - C AZ - D AZ - E Asia Pacific (Tokyo) AZ - A AZ - B AZ - C
  7. 7. Achieving High Availability Using Multi-AZ Availability Zone - A Availability Zone - B Availability Zone - C Region
  8. 8. AWS Taiwan Customers
  9. 9. AWS Taiwan Customers
  10. 10. AWS Taiwan Customers
  11. 11. AWS 大數據分析服務 Amazon EMR Amazon Elasticsearch Amazon Kinesis Amazon Redshift Amazon Quicksight Amazon Machine Learning Hadoop, Spark, HBase, Hive, Presto, Mahout, Pig, Zeppelin Elasticsearch 即時串流資料 資料倉儲 商業智慧 機器學習
  12. 12. AWS 大數據分析服務 Amazon EMR Amazon Elasticsearch Amazon Kinesis Amazon Redshift Amazon Quicksight Amazon Machine Learning Hadoop, Spark, HBase, Hive, Presto, Mahout, Pig, Zeppelin Elasticsearch 即時串流資料 資料倉儲 商業智慧 機器學習 Amazon Athena 使用標準的SQL語法 分析儲存在 Amazon S3的資料
  13. 13. Architecture
  14. 14. Availability Zone Private SubnetPublic Subnet Availability Zone Private SubnetPublic Subnet Remote Users Sample Microsoft Architecture Virtual Private Gateway Corporate Office IIS App IIS Web IIS App IIS Web VPN AWS Direct Connect Internet Gateway RDGW VPC NAT Gateway RDGW VPC NAT Gateway AWS Directory Service AWS Directory Service MS SQL MS SQL Always On Availability Group VPC Endpoint Amazon S3 Auto Scaling
  15. 15. Secure remote administration architecture Availability Zone Gateway Security Group Web Security Group Private SubnetPublic Subnet Accept TCP Port 443 from Admin IP Accept traffic from Gateway SG AWS Administrator Corporate Data Center WEB2 TCP 443 WEB1 RDGW Requires one connection: • Connect to the RD Gateway, and the gateway proxies the RDP or PowerShell connection to the back- end instance.
  16. 16. Deploying SQL Server on AWS Choosing the Best Option for Your Needs
  17. 17. Choose the Best Option for Your Needs
  18. 18. Choose the Best Option for Your Needs ü Managed physical infrastructure ü Managed OS installation ü Managed scaling ü OS-level control ü Managed physical infrastructure ü Managed DB installation and backups ü Managed OS and patching ü Managed high availability and scaling
  19. 19. Your Responsibility v App optimization, tuning v Deployment v Monitoring v High availability v Backups v DB & OS patching v App optimization, tuning v Deployment v Monitoring
  20. 20. Amazon RDS for SQL Server Consider RDS first Focus on: • Business value tasks • High-level tuning tasks • Schema optimization No in-house database expertise Which Option Is Right for You? SQL Server on Amazon EC2 Need control over: • DB instance & OS • Backups, Replication • Clustering • sysadmin role Use options not in Amazon RDS
  21. 21. SQL Server Features at a Glance * Self-installed Amazon RDS for SQL Server SQL Server on Amazon EC2 Versions Supported: 2008 R2, 2012, 2014, 2016 2005*, 2008*, 2008 R2, 2012, 2014, 2016 Editions Supported: Express, Web, Standard, Enterprise High Availability: Self-managed; AlwaysOn, Mirror, Log ShipAWS-managed Encrypted storage using AWS KMS (all editions); TDE supportEncryption: Authentication: Windows & SQL authentication Maintenance plans & third-party toolsManaged automated backupsBackups: Self-managedAutomatic software patchingMaintenance:
  22. 22. License Included • Available for Amazon RDS • Use an Amazon Machine Image (AMI) that includes SQL Server for use on Amazon EC2 • Licensing cost included in the hourly cost of the EC2 instance or RDS DB instance • Available for Web, Standard, and Enterprise editions Licensing Options for SQL Server on AWS Bring Your Own License • Amazon RDS and Amazon EC2 in default tenancy require License Mobility through Software Assurance • Can use per-core or per-socket licenses with Amazon EC2 Dedicated Hosts without License Mobility benefit • License Mobility requires benefit verification with Microsoft
  23. 23. SQL Server on Amazon EC2 Best Practices
  24. 24. SQL Server Best Practices on Amazon EC2 • Getting the most out of AWS storage options • Configure tempdb with multiple files on instance storage (or fast Amazon EBS storage if instance storage is unavailable) • Availability Zones and AlwaysOn Availability Groups: achieving both HA and DR with just two servers • Failover cluster instances: I Get By With a Little Help From My Friends • Instant file initialization
  25. 25. Amazon Elastic Compute Cloud (EC2) Resizable compute capacity Complete control of your computing resources Reduces the time required to obtain and boot new server instances to minutesAmazon EC2
  26. 26. Instances and AMIs Select an AMI based on: Region Operating system Architecture (32-bit or 64-bit) Launch permissions Storage for the root device AMI Instances Instance Launch instances of any type Host computer Host computer
  27. 27. Amazon EC2 Instances OS, Applications, & Configuration AMI Running or Stopped VM Instances AZ VPC Region EBS S3 EBS Snapshots S3 Buckets EBS EBS EBS EBS EBS AZ Instances Instances
  28. 28. Amazon EBS vs. Amazon EC2 Instance Store Amazon EBS • Data stored on an Amazon EBS volume can persist independently of the life of the instance. • Storage is persistent. Amazon EC2 Instance Store • Data stored on a local instance store persists only as long as the instance is alive. • Storage is ephemeral.
  29. 29. AMI Types - Storage for the Root Device Characteristic Amazon EBS-Backed Amazon Instance Store-Backed Boot time Usually < 1 minute Usually < 5 minutes Size limit 16 TiB 10 GiB Data persistence The root volume is deleted when the instance terminates. Data on any other Amazon EBS volumes persists after instance termination. Data on any instance store volumes persists only during the life of the instance. Charges Instance usage, Amazon EBS volume usage, and storing your AMI as an Amazon EBS snapshot. Instance usage and storing your AMI in Amazon S3. Stopped state Can be stopped. Cannot be stopped.
  30. 30. Instance Lifecycle AMI pending Launch runningrebooting Reboot Start terminated shutting-down Terminate Terminate EBS-backed instances only Stop stopping stopped
  31. 31. Choosing the Right Amazon EC2 Instance EC2 instance types are optimized for different use cases and come in multiple sizes. This allows you to optimally scale resources to your workload requirements. AWS uses Intel® Xeon® processors for EC2 instances, providing customers with high performance and value. Consider the following when choosing your instances: Core count, memory size, storage size and type, network performance, and CPU technologies. Hurry Up and Go Idle - A larger compute instance can save you time and money, therefore paying more per hour for a shorter amount of time can be less expensive.
  32. 32. Amazon EBS Lifecycle Vast amounts of unused space Create Call CreateVolume 1 GB to 16 TB Attach Call AttachVolume to affiliate with one Amazon EC2 instance Attached and In Use • Format from Amazon EC2 instance OS • Mount formatted drive CreateSnapshot Snapshot to Amazon S3 Detach Call DetachVolume Deleted Call DeleteVolume
  33. 33. Amazon EBS and Amazon S3 Amazon EBS Amazon S3 Paradigm Block storage with file system Object store Performance Very fast Fast Redundancy Across multiple servers in an Availability Zone Across multiple facilities in a Region Security EBS Encryption – Data volumes and Snapshots Encryption Access from the Internet? No (1) Yes (2) Typical use case It is a disk drive Online storage (1) Accessible from the Internet if mounted to server and set up as FTP, etc. (2) Only with proper credentials, unless ACLs are world-readable
  34. 34. Amazon Elastic Block Storage What is Amazon Elastic Block Storage (EBS)? • Network-attached block storage • Available for all instance types • Many instance types support EBS optimization – dedicated channel for network storage I/O, eliminating contention with regular I/O • Some instance types are EBS optimized, others offer it as an option
  35. 35. Amazon EBS Volume Types Volume Type General Purpose: GP2 Provisioned IOPS: PIOPS/IO1 Throughput Optimized: ST1 Cold HDD: SC1 Technology: SSD SSD Magnetic Magnetic Sizes: 1 GiB – 16 TiB 4 GiB – 16 TiB 500 GiB – 16TiB 500 GiB – 16 TiB Max. IOPS: 10,000 20,000 500 250 Max. Throughput: 160 MiB/sec 320 MiB/sec 500 MiB/sec 250 MiB/sec Properties: 3 IOPS/1 GB, burstable up to 3000 IOPS for max 1 TiB volumes Consistent provisioned performance, up to 50 IOPS/GB Optimized for throughput, and sequential read/write workloads, baseline per TiB throughput, with burst capability
  36. 36. Amazon EC2 Instance Storage What is instance storage? • Some instance types come with direct attached disk-based storage • Included in the hourly cost • Data on instance storage does not persist a user- initiated instance stop/start or hardware failure • Must be allocated at launch • Fast disk I/O without going over the network
  37. 37. Storage Performance for EC2 SQL Server Consider IOPS and throughput needed by your workload • Enable EBS optimization on instance • Create a single volume for data and logs • Format with 64K allocation unit size • Match total EBS IOPS and throughput to instance type • Stripe EBS PIOPS volumes for more than 20,000 IOPS Example volume layout: C: Boot on General Purpose SSD D: Data and log files on PIOPS single or striped set E: Backups on ST1 or SC1 Z: Tempdb on instance storage (if available)
  38. 38. Configuring tempdb on Instance Storage Move tembdb files to instance-storage-backed drives:1 2 ALTER DATABASE tempdb MODIFY FILE (NAME = tempdev, FILENAME = 'Z:tempdb.mdf'); GO ALTER DATABASE tempdb MODIFY FILE (NAME = templog, FILENAME = 'Z:templog.mdf'); GO Modify startup to grant service account access: icacls Z: /grant "NT SERVICEMSSQLSERVER”:(OI)(CI)(F)
  39. 39. More tempdb Optimization Options Consider: • Using multiple tempdb files (1:1 mapping with CPUs, up to 8) • Striping multiple instance storage disks together for higher I/O • Changing SQL Server service startup to Automatic (Delayed Start) to allow instance storage to provision • Scripting/automating configuration on instance boot Striping solution by consulting partner IFM Ltd.
  40. 40. SQL Server HA & DR on Amazon EC2 Use multiple Availability Zones • Instance-level and AZ-level failure tolerance • Synchronous replication Options • Enterprise Edition: AlwaysOn Availability Groups • Standard Edition: Failover cluster instances using partner block-level replication solution
  41. 41. Amazon Virtual Private Cloud (VPC) Provision a private, isolated virtual network on the AWS cloud. Have complete control over your virtual networking environment. Amazon VPC
  42. 42. VPCs and Subnets A subnet defines a range of IP addresses in your VPC. You can launch AWS resources into a subnet that you select. A private subnet should be used for resources that won’t be accessible over the Internet. A public subnet should be used for resources that will be accessed over the Internet. Each subnet must reside entirely within one Availability Zone and cannot span zones.
  43. 43. Amazon VPC Example Availability Zone A Virtual Private Cloud AWS Cloud Public Subnet Internet Virtual Private Cloud Availability Zone B Private Subnet Availability Zone C VPN Only Subnet DB Server DB Server App Server DB Server DB Server DB Server Web Server Web Server NAT Customer Network R
  44. 44. Security in Your VPC Security groups Network access control lists (ACLs) Subnet Internet GatewayVPN Gateway VPC Router Security Group Security Group Security Group Network ACL Network ACL Routing Table Routing Table Instance Instance Instance Instance Subnet
  45. 45. VPN Connections VPN Connectivity option Description AWS Hardware VPN You can create an IPsec, hardware VPN connection between your VPC and your remote network. AWS Direct Connect AWS Direct Connect provides a dedicated private connection from a remote network to your VPC. AWS VPN CloudHub You can create multiple AWS hardware VPN connections via your VPC to enable communications between various remote networks. Software VPN You can create a VPN connection to your remote network by using an Amazon EC2 instance in your VPC that’s running a software VPN appliance.
  46. 46. Multi-AZ AlwaysOn Availability Group Availability Zone 1 Private Subnet EC2 Primary Replica Availability Zone 2 Private Subnet EC2 Secondary Replica AWS Region Synchronous Commit Automatic Failover
  47. 47. Multi-region AlwaysOn Availability Group Availability Zone 1 Private Subnet EC2 Primary Replica Primary: WSFC: AG Listener: AWS Region A Availability Zone 2 Private Subnet EC2 Secondary Replica Primary: WSFC: AG Listener: Availability Zone 1 Private Subnet EC2 Secondary Replica Primary: WSFC: AG Listener: AWS Region B Elastic IP Elastic IP VPN Synchronous Commit Automatic Failover Asynchronous Commit Manual Failover
  48. 48. Failover Cluster Instance Amazon EBS Amazon EBS Availability Zone 1 Private Subnet EC2 Primary Node Availability Zone 2 Private Subnet EC2 Secondary Node AWS Region Data Replication SoftNAS / SIOS
  49. 49. SQL Server Instant File Initialization What is database file initialization? • Normally, database and log files are initialized to overwrite leftover disk data • File initialization causes some DB operations to take longer • Instant database file initialization reclaims unused disk space without zeroing it out
  50. 50. Instant File Initialization Security Concerns • Deleted content is overwritten only when new data is written to file • Deleted content might be accessible by an unauthorized principal • Disclosure threat is reduced while the DB file is attached to the SQL Server instance Mitigations: • Apply restrictive discretionary ACLs on data files and backup files • Disable instant file initialization
  51. 51. SQL Server 2016 Install Time Enabling Instant Database File Initialization Post-Install or Other Versions Grant Perform volume maintenance tasks to SQL Server service account 1. Open the Local Security Policy app, 2. From Local Policy, choose User Rights Assignment. 3. Double-click Perform volume maintenance tasks. 4. Choose Add User or Group. us/library/ms175935.aspx
  52. 52. Amazon RDS for SQL Server Best Practices
  53. 53. Amazon RDS for SQL Server Best Practices • Moving/migrating data from Amazon RDS • Leveraging SQL Server’s native .bak backup and restore • Using highly available SQL Server deployments in Amazon RDS • Managing SQL Server storage and I/O performance • Leveraging existing Active Directory with Amazon RDS for SQL Server
  54. 54. Moving Data In and Out of RDS for SQL Server .BAK File Import and Export Leverages SQL Server’s native backup functionality AWS Database Migration Service Minimize downtime during migrations, migrate between different DB platforms, Schema Conversion Tool AWS Marketplace Third-party data import and export tools and solutions 1 3 4 Microsoft SQL Server Database Publishing Wizard, Import/Export Export to T-SQL files, load using sqlcmd 2
  55. 55. .bak File Import and Export Prerequisites RDS for SQL Server DB Instance✓ S3 Bucket (to store .bak files)✓ DB Option Group enabling SQLSERVER_BACKUP_RESTORE✓ SSMS or other client to connect to DB instance and execute the stored procedures ✓
  56. 56. Using .bak File Import and Export /* Restoring from backup file */ exec msdb.dbo.rds_restore_database @restore_db_name='your database name' @s3_arn_to_restore_from='arn:aws:s3:::<bucket>/<file path>'; /* Exporting to backup file */ exec msdb.dbo.rds_backup_database @source_db_name='your database name', @s3_arn_to_backup_to='arn:aws:s3:::<bucket>/<file path>', @overwrite_S3_backup_file=1; /* Check job status */ exec msdb.dbo.rds_task_status;
  57. 57. High Availability in RDS for SQL Server Amazon RDS for SQL Server Multi-AZ • Principal and secondary DB nodes in different Availability Zones • Leverages SQL Server DB mirroring • Automatic failover (typically, 1–2 minutes) • Always run production workloads in Multi-AZ mode
  58. 58. Amazon RDS Multi-AZ in-Depth Failure scenarios mitigated: • Loss of availability in primary AZ • Loss of network connectivity to principal DB node • Compute unit or storage failure on principal DB node Failover process: Consider: • Implementing retry logic at the application layer—trigger manual failover to test • Impact on mirroring of changing heavy workloads (for example, index rebuilds) Mirroring stopped Address apply debt Promote to master Change DNS endpoint Provision new secondary
  59. 59. Storage I/O Performance Amazon RDS Amazon EC2 Type Size Performance Size Performance Burst Capacity Pricing Model Magnetic Storage 20 GiB–1 TiB ~100 IOPS 1 GiB–1 TiB ~ 100 IOPS Yes, several hundred IOPS Allocated storage; I/O operations General Purpose (SSD) 20 GiB–4 TiB (min. 100 GiB recommended) 3 IOPS/GiB 1 GiB–16 TiB 3 IOPS/GiB for volumes 1 TiB or less, up to 10,000 IOPS for larger volumes Yes, up to 3000 IOPS per volume, subject to credits (< 1 TiB in size) Allocated storage Provisioned IOPS (SSD) 100 GiB–4 TiB (min. 200 GiB for Standard edition and up) Up to max. 20,000 IOPS 4 GiB–16 TiB Up to 20,000 IOPS No, fixed allocation Allocated storage; provisioned IOPS
  60. 60. Storage I/O Performance Planning Amazon RDS storage throughput depends on DB instance class I/O requests sizes: Provisioned IOPS can handle I/O up to 256 KB in size I/Os larger than 32 KB consume multiple IOPS Maximum storage IOPS: 20,000 Capacity for concurrent I/O–optimize latency 1 3 2 Average queue depth: I/O requests waiting to be serviced ~5 outstanding I/O op/1000 IOPS provisioned 4
  61. 61. Existing Active Directory Integration • Windows Authentication support provided by AWS Directory Service Microsoft AD directory • RDS DB instance joined to the directory operated domain • Integrate with existing AD deployment using a forest trust • Configure inbound trust on the external forest + outbound trust in the directory • Configure conditional forwarders for the two domains
  62. 62. Thank you!
  63. 63. Useful Resources Microsoft SQL Server on AWS Deploying SQL Server on AWS (whitepaper) Amazon RDS for SQL Server Supported Features Implementing Microsoft Windows Server Failover Clustering and SQL Server AlwaysOn Availability Groups in the AWS Cloud