Design, Deploy, and Optimize Microsoft SQL Server on AWS (WIN324-R1) - AWS re:Invent 2018

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Design, Deploy, and Optimize Microsoft SQL Server Workloads on AWS Wojtek Piaseczny Sr. Manager, Amazon RDS Amazon Web Services Veronika Durgin Manager, Data Systems Sonos W I N 3 2 4 - R

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What sets AWS apart? Building and managing cloud since 2006; millions of active customers; rapid customer-driven releases Experience & innovation Quickly spin up; scale up or down as neededAgility, elasticity, & scalability 19 Regions, 57 Availability Zones, 119 Edge Points of PresenceGlobal footprint Tens of thousands of partners; 4,000+ AWS Marketplace productsPartner community #1 priority; fine-grained control; certifications/accreditationsSecurity Economies of scale; 65 price drops since inceptionCost savings

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Innovation for Windows on AWS KeyWindowslaunchessince2008 90+ 40+ 750+ instance types, 22 instance families different AMIs for Windows workloads Windows ISV listings in AWS Marketplace Windows Deep Learning AMI Hyper-V support in SMS Application-consistent snapshots through VSS WS 2008 & SQL Server 2008 Visual Studio Toolkit Microsoft SCOM plugin release AWS Directory Service Amazon EC2 Dedicated Hosts (BYOL) Microsoft SharePoint 2016 (Marketplace) Windows Server 2008 R2 SQL Server 2008 R2 Windows Server 2003 SQL Server 2005 .NET SDK Microsoft SCVMM plugin Windows Server 2012 SQL Server 2012 AWS Tools for Windows PowerShell Amazon RDS adds SQL Server EC2 Run Command Amazon EC2 Systems Manager Windows Server & SQL Server 2016 Amazon EC2 Dedicated Instances (BYOL) .NET on AWS Lambda & AWS CodeBuild SAP instance on AWS 2012 AWS Trusted Advisor checks for Windows SQL Server 2017 Windows & Linux X-Ray .NET SDK Customeradoption Windows for Lightsail .NET Core & PowerShell on AL2/Ubuntu 2008 Today .NET Core 2.0 Support with Lambda & AWS X-Ray

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Where can I run my SQL Server workloads on AWS? Amazon Relational Database Service (Amazon RDS) Amazon Elastic Compute Cloud (Amazon EC2) Managed service with up to 64 vCPU, 488 GB RAM, and 16 TB storage Self-managed virtual machine with up to 128 vCPU, 4 TB RAM, and 400 TB storage

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS for SQL Server SQL Server on Amazon EC2 Power, HVAC, net OS install/maintenance OS patching DBMS install/maintenance DBMS patching Database backups High availability Scaling AWS managed Customer managed Power, HVAC, net OS install/maintenance OS Patching DBMS install/maintenance DBMS Patching database backups High availability Scaling • Consider RDS first • Focus on business value tasks • High-level tuning • Schema optimization • No in-house database expertise • Automatic host replacement • Need full control over DB instance • Replication • Clustering • Options that are not available in Amazon RDS Options for deploying SQL Server on AWS

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SQL Server 2017 on Linux Benefits Multi-Linux distribution support (for example, Amazon Linux, Ubuntu, RHEL) Supports containers Flexibility – Language support (for example, Java, Ruby) Performance – Fast queries Security – Reduced attach surface Cost – No OS licensing Simplicity – Ease of migration SQL Server Editions Express Web Standard Enterprise Available AMIs

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EC2: Purpose-built compute families Current instance families and generation Family/usage M5, M4 General purpose compute T2 Burstable performance C5, C4 Compute optimized X1, X1E, R5, R5d, R4, R3 Memory optimized P2, G3, F1 Accelerated computing I3 Storage optimized (I/O) D2 Storage optimized (density)

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EC2: Scaling to meet your needs Model vCPU Mem (GiB) Networking perf. SSD storage (GB) r5d.large 2 16 Up to 10 gigabit 1 x 75 NVMe SSD r5d.xlarge 4 32 Up to 10 gigabit 1 x 150 NVMe SSD r5d.2xlarge 8 64 Up to 10 gigabit 1 x 300 NVMe SSD r5d.4xlarge 16 128 Up to 10 gigabit 2 x 300 NVMe SSD r5d.12xlarge 48 384 10 gigabit 2 x 900 NVMe SSD r5d.24xlarge 96 768 25 gigabit 4 x 900 NVMe SSD

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. License optimization with Optimize CPUs • Control active vCPUs and hyper-threading status when launching new EC2 instances • Reduce the number of SQL Server licenses *Sample licensing example only Instance type Total vCPUs Active vCPUs with Optimize CPUs SQL Server license savings r5.4xlarge 16 8 50% r5.12xlarge 48 8 83%

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do I use Optimize CPU? Set with AWS CLI run-instances --cpu-options “CoreCount=x,ThreadsPerCore=y” View with AWS CLI describe-instances “CpuOptions”: {“CoreCount”: x, “ThreadsPerCore”: y} Alternatively, set with AWS SDK or Amazon EC2 API

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customer reference “With Optimize CPUs our customers, such as a research firm that specializes in research for the financial services industry, can not only migrate their existing Microsoft SQL Licenses to Amazon EC2 but do so on instances that are ideally tuned for Memory, CPU, IO, and Bandwidth while not having to overprovision CPU cores or acquire additional Microsoft SQL Licenses, providing a savings of 50% on their licensing.” Patrick Hannah Vice President of Engineering

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS has a variety of storage options Amazon EC2 instance store (ephemeral volumes) Amazon Elastic Block Store (Amazon EBS) Amazon Elastic File System (Amazon EFS) Amazon Simple Storage Service (Amazon S3) Amazon Glacier AWS Storage Gateway: File Gateway AWS Snowball & AWS Snowball Edge AWS Snowmobile

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Data/logs EC2 Instance Store sc1st1 io1gp2 EBS SSD-backed volumes EBS HDD-backed volumes I2/I3 D2/H1 Database storage Backups AmazonS 3

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EBS • Persistent block-level storage for Amazon EC2 • Pay only for what you provision • Native redundancy and write cache • Consistent and low-latency performance • Optimized for random I/O • Native support for encryption at rest (data volumes) • Adjust performance and price on the fly GP2 IO1 Most Workloads Critical Workloads Predictable burst and baseline I/O intensive workloads

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EC2 instance sizing for EBS-optimized m4.large 2 vCPU 8-GiB RAM Dedicated to Amazon EBS 450 Mbps ~ 56.25 MiB/s 2 TiB GP2 volume: 6,000 IOPS 160 MiB/s max throughput 3,600 16K IOPS

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EC2 Instance sizing for EBS-optimized m4.large 2 vCPU 8-GiB RAM 450 Mbps ~ 56.25 MiB/s 2 TiB GP2 volume: 6,000 IOPS 160 MiB/s max throughput 3,600 16K IOPS m4.2xlarge 8 vCPU 32-GiB RAM Dedicated to EBS 1 Gbps ~ 125 MiB/s 8,000 16K IOPS 2 TiB GP2 volume: 6,000 IOPS 160 MiB/s max throughput Dedicated to Amazon EBS

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is Amazon EC2 instance store? • Local to instance • Non-persistent data store • SSD or HDD • Data not replicated (by default) • No snapshot support EC2 instances Physical host Instance store or * Not all instance types have local, instance storage

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon EC2 instance store use cases Temporary storage (SQL Server TempDB) R5d instance type • Temporary storage (SQL Server TempDB) • Very low latency (0.1 ms) • Extremely high IOPS (over 3 million vs. 80K for EBS-optimized) • Higher throughput (almost 10x EBS max) D2 instance type • Up to 48 TB of HDD-based local storage

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Databases on Amazon EBS: Storage best practices • Local, instance storage only for temp data • io1 for critical workloads • st1 for backups (along with Amazon S3 & Amazon Glacier) “We’ve seen much stronger performance for our database-backup workloads with the Amazon EBS st1 volumes, and we’re also saving 75% on our monthly backup costs.” Randy Young, Director of Cloud Operations, Infor

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ Always On availability group Availability Zone 1 Private subnet Availability Zone 2 Private subnet AWS Region Amazon EC2 primary replica Amazon EC2 secondary replica Synchronous commit automatic failover

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-region Always On Availability Group Availability Zone 1 Private subnet AWS Region A Availability Zone 2 Private subnet Availability Zone 1 Private subnet AWS Region B Elastic IP Elastic IP VPN Amazon EC2 primary replica Amazon EC2 secondary replica Amazon EC2 secondary replica Synchronous commit automatic failover Asynchronous commit manual failover

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-region distributed availability group Availability Zone 1 Private subnet AWS Region A Availability Zone 2 Private subnet Amazon EC2 primary replica Amazon EC2 secondary replica Synchronous commit automatic failover Availability Zone 1 Private subnet AWS Region B Availability Zone 2 Private subnet Amazon EC2 primary replica Amazon EC2 secondary replica Synchronous commit automatic failover VPC peering Asynchronous commit manual failover

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ failover cluster instance Amazon EBS Amazon EBS Availability Zone 1 Private subnet Availability Zone 2 Private subnet AWS Region Amazon EC2 primary replica Amazon EC2 secondary replica Data replication

30. Veronika – Sonos

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. It’s hard “Nothing in the world is worth having or worth doing unless it means effort, pain, difficulty …” – Theodore Roosevelt

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SQL Server Witness (B)AG Failover cluster Log shipping Tran logs SQL Server bottlenecks are generally memory and I/O “Relying on servers not to crash is called luck, not a DR strategy.” – Unknown Test backup restores/run DBCC

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. First attempt R3.* Amazon EC2 type • Provisioned IOPs Amazon EBS for data • Z drive (instance store) for TempDB • AG with SQL Enterprise What we’ve learned • Expensive • IOPs matter • Amazon EBS optimization throughput • AWS IAM role for Amazon S3 • Tight security groups Test, learn, and iterate

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ephe … Epheme … instance store • I3.* • Pros: • Faster • Cheaper (1/3 cost: I3.4xl vs R3.4XL with 40K EBS) • Cons: • Instance store is gone on stop/start (not reboot) • Other cool stuff • SQL Server 2016 Standard sp1 BAG • Core optimized

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Current setup • I3.* • SQL Server 2016 sp1 + • (B)AG • SQL installed on C: drive • TempDB and user DBs on ephemeral • Near 0 latency Witness (B)AG Amazon S3 Restore/DBCC daily

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo summary R MB/s R IOPs R avg Lat W MB/s W IOPs W avg Lat R4 I3 R4 I3 R4 I3 R4 I3 R4 I3 R4 I3 Rand 8K 101 947 12975 121148 2.4 0.3 101 381 12974 48822 2.5 0.6 Rand 64K 101 957 1622 15310 20 2.1 101 381 1622 6102 20 5.2 Rand 512K 101 953 202 1907 158 16.8 101 381 405 1526 79 20.1 Seq 8K 101 381 12975 48822 2.5 0.7 Seq 64K 101 381 1621 6102 20 5.2 • I3.xl vs R4.xl (1 TB 20K IO1 Amazon EBS) • Quicker reads, quicker writes, low latency • Diskspd • -t4 (threads), -o8 (Outstanding Request), -d120 (2 min), 20 GB file

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Other configurations • I3.* • AG both ENT and STD • Archive/barely used DBs on GP2 Amazon EBS volumes • Core optimization (disable hyper-threading/disable cores) • Distributor • M3.* with reserved IOPs drives for distribution DB • SSAS • X1E.* • Amazon S3 for backups • We manage deletes (not through Amazon S3 lifecycle policy)

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Future “Once you achieve one goal, you should be looking forward to trying to build onto the next thing, and not just getting comfortable with what you're doing.” – LL Cool J

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tips and tricks for Amazon EC2 • Test, test, test! • High availability for production workloads • All databases on same primary node in AlwaysOn • Dedicated ENI for replication traffic • Backups, audit logs to Amazon S3 • Nightly backups, incremental log backups for PITR • Storage backups vs. SQL Server backups • Newer instance types are generally faster and cheaper

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ SQL Server on Amazon RDS Availability Zone 1 Private subnet Availability Zone 2 AWS Region Amazon RDS primary Amazon RDS secondary Private subnet Automatic failover

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ SQL Server on Amazon RDS Availability Zone 1 Private Subnet Availability Zone 2 AWS Region Amazon RDS Primary Amazon RDS Secondary Private Subnet Automatic FailoverManaged service

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tips and tricks – Amazon RDS • Test, test, test! • High availability for production workloads • All databases on same primary node in AlwaysOn • Dedicated ENI for replication traffic • Backups, audit logs to Amazon S3 • Nightly backups, incremental log backups for PITR • Storage backups vs. SQL Server backups • Newer instance types are generally faster and cheaper • Amazon RDS has hundreds of thousands of customers, use that learning to continually improve

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are AWS DMS and AWS SCT? AWS Database Migration Service (AWS DMS) easily and securely migrates and/or replicates your databases and data warehouses to AWS AWS Schema Conversion Tool (AWS SCT) converts your commercial database and data warehouse schemas to open-source engines or AWS- native services, such as Amazon Aurora and Amazon Redshift Over 90,000 database instances migrated and counting …

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. When to use AWS DMS and AWS SCT? Modernize Migrate Replicate Modernize your database tier – • Commercial to open source • Commercial to Amazon Aurora Modernize your data warehouse – • Commercial to Amazon Redshift • Migrate business-critical applications • Migrate from Classic to VPC • Migrate data warehouse to Amazon Redshift • Upgrade to a minor version • Consolidate shards into Aurora • Create cross-region read replicas • Run your analytics in the cloud • Keep your dev/test and production environment in sync

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Monitoring SQL Server performance Monitor performance using Amazon CloudWatch Alarms & notifications: Amazon RDS & Amazon EC2 Default metrics: Amazon RDS & Amazon EC2 Custom metrics: Amazon EC2 1 2 3 CPU utilization Read/write IOPS Disk queue depth Memory (RDS) Storage space (RDS) Connections (RDS) I/O throughput (EC2) … Use Extended Events (SQL Server Profiler) & Tuning Advisor to trace query performance Database log files: Amazon RDS & Amazon EC24

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. In-depth monitoring Amazon RDS enhanced monitoring • OS level monitoring metrics – 26 system and per-process metrics • Metrics delivered to Amazon CloudWatch Logs • Up to 1-second granularity Amazon EC2 unified CloudWatch agent • OS level monitoring metrics – 80+ system and per-process metrics • Metrics delivered to CloudWatch Logs Compared to CloudWatch metrics: • Agent-based metrics collections • There can be differences with CloudWatch metrics due to collection source (hypervisor vs. agent) – for example, CPU

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database security AWS manages Customer manages • Physical infrastructure controls • Host access (Amazon RDS) • Operating system version/patches (Amazon RDS) • How database patches are applied (Amazon RDS) • Database users and privileges (Windows Authentication) • Access to data (TDE, Always Encrypted) • SQL audit • Network controls (VPC security groups)

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Encryption App servers Storage • At rest • Transparent data encryption • Amazon EBS storage encryption • In transit – SSL • Force SSL for all connections • Per-connection SSL

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Auditing AWS API auditing (CloudTrail) Database auditing • Audit user API calls (Create-, Modify-, DeleteDBInstance) • Integrates with Cloudwatch Logs and CloudWatch Events • Stored in Amazon S3 • Server-level or database-level audits • Retention managed by customer • Logs stored on where you want them (on host, Amazon S3)

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compliance •Assurance programs for finance, healthcare, government, & more •Shared responsibility model •AWS → Security of the cloud •Customer → Security in the cloud •https://aws.amazon.com/compliance/

Design, Deploy, and Optimize Microsoft SQL Server on AWS (WIN324-R1) - AWS re:Invent 2018

You are reading a preview.

Check these out next

Design, Deploy, and Optimize Microsoft SQL Server on AWS (WIN324-R1) - AWS re:Invent 2018

More Related Content

Slideshows for you (20)

Similar to Design, Deploy, and Optimize Microsoft SQL Server on AWS (WIN324-R1) - AWS re:Invent 2018 (20)

More from Amazon Web Services (20)