Phissug s01 ep6, stretch database
•Download as PPTX, PDF•
0 likes•599 views
A short presentation filled with screenshots to get you started on stretching your database to Microsoft Azure cloud services.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Phissug s01 ep6, stretch database
Similar to Phissug s01 ep6, stretch database (20)
Recently uploaded
Recently uploaded (20)
Phissug s01 ep6, stretch database
- 1. PHISSUG Sessions Season 1, Episode 6: SQL Server in the Cloud
- 2. Argelo Royce P. Bautista 4 years in the it industry Full-stack developer Enjoys creating reports (SSRS, excel, & power query) Enjoys exploring Microsoft technologies Enjoys learning and knowledge sharing with the tech community
- 3. Hybrid Will Be the Most Common Use of the Cloud Gartner Says By 2020, a Corporate "No-Cloud" Policy Will Be as Rare as a "No- Internet" Policy Is Today By 2019, more than 30 percent of the 100 largest vendors' new software investments will have shifted from cloud-first to cloud-only. By 2020, more compute power will have been sold by IaaS and PaaS cloud providers than sold and deployed into enterprise data centers. The Infrastructure as a Service (IaaS) market has been growing more than 40 percent in revenue per year since 2011, and it is projected to continue to grow more than 25 percent per year through 2019. By 2019, the majority of virtual machines (VMs) will be delivered by IaaS providers. By 2020, the revenue for compute IaaS and Platform as a Service (PaaS) will exceed $55 billion — and likely pass the revenue for servers. Reference: http://www.gartner.com/newsroom/id/3354117
- 4. “ ” Stretch feature of SQL Server 2016 A new way to archive your data. Disclaimer: Majority of the parts of the presentation were taken from Joe Yong’s presentation of Stretch Db and Microsoft Introduction of SQL Server 2016’s new features at the Microsoft Ignite 2015 The videos of StretchDb and AlwaysEncrypted were taken from Microsoft’s New SQL. No Equal Series @ https://channel9.msdn.com
- 5. Agenda 1. Introduction 2. Use cases 3. Enabling Stretch on Database 4. Stretching the Table 5. Disable stretching 6. migrate data that satisfies condition 7. Use function to migrate data that satisfy condition 8. Show Trickling and spaces used 9. Migrating child rows from related tables 10. Querying the stretch table 11. Querying related stretched tables 12. Backup and Restore 13. Enable always encrypted 14. Stretching temporal tables 15. Pricing
- 7. Capability Stretch large operational tables from on-premises to Azure with the ability to query Benefits BI integration for on-premises and cloud Cold/closed data Orders In-memory OLTP table Hot/active data Order history Stretched table Trickle data movement and remote query processing On-premises Azure Stretch SQL Server into Azure Securely stretch cold tables to Azure with remote query processing
- 8. Databases today Business users and decision makers Want or need to retain cold data for many years (indefinitely) Cold data must be online SAN consumption increasing faster than IT budgets Want to access cold data from same application Database administrators Uncontrollable growth in size and sprawl Operations windows exceeding SLA (index, backup/restore) Frequent storage acquisition and provisioning Users can’t/won’t say what data can be deleted TCO PerfData access Cost Performance ■ Current Solutions Admin overhead
- 9. Keep, keep, keep! Cut, cut, cut! ??? !#$^* Business owners, end-users, … Storage admins, budget owners, … DBA
- 10. StretchDB considerations No change to access controls, DBA processes, applications, tools, operationsGoals Cold data always online Significantly lower storage TCO Access cold data with existing applications Easier performance and index maintenance Faster backup/restore Automatically managed and protected cold data Tradeoffs Moderate performance reduction for cold data access Update and delete on cold data is an administrative function Some functional limitations TCO PerfData access ■ Stretch Database ■ Current Solutions Cost PerformanceAdmin overhead
- 11. StretchDB – enabling AzureOn-premises On- premises SQL Server instanceOn-premises application(s) DB in SAN/Local Storage Ord_Detail Storage Ord_detail_ archive table Txn_detail Hot + cold rows Compute
- 22. StretchDB – Enabling (Table)
- 23. StretchDB – Enabling (Table)
- 24. StretchDB – Enabling (Table)
- 25. StretchDB – Enabling (Table)
- 26. StretchDB – Enabling (Table)
- 27. Enabling tables and specifying criteria for cold data (Filter via GUI)
- 28. Enabling tables and specifying criteria for cold data (Filter via GUI)
- 29. Enabling tables and specifying criteria for cold data (Filter via TSQL using Function)
- 30. StretchDB – data movement Azure On- premises SQL Server instanceOn-premises application(s) DB in SAN/Local Storage Ord_Detail Storage Ord_detail _archive table Trickle data migration Ord_detail_ archive table Txn_detail Txn_detail (cold rows only) Hot + cold rows in same table Compute Entire archive table moved Only cold rows moved On-premises
- 31. StretchDB – Data Movement
- 32. StretchDB – Data Movement
- 33. StretchDB – Data Movement
- 34. StretchDB – Data Movement
- 35. StretchDB – Data Movement
- 36. StretchDB – Data Movement
- 37. StretchDB – Data Movement Before After
- 38. StretchDB – smart query processing Azure On- premises SQL Server instanceOn-premises application(s) DB in SAN/Local Storage Ord_Detail Storage Ord_detail _archive table Ord_detail_ archive table Txn_detail Txn_detail (cold rows only) Transparent remote data access Hot + cold rows in same table Compute Local data only queries Local + remote data queries Trickle data migration On-premises
- 39. StretchDB – Smart Query Processing
- 40. StretchDB – Smart Query Processing
- 41. Parent-Child Relationship (Child is a Stretched Table)
- 42. Parent-Child Relationship (Child is a Stretched Table)
- 43. Parent-Child Relationship (Child is a Stretched Table)
- 44. StretchDB – backup & restore Azure On- premises SQL Server instanceOn-premises application(s) DB in SAN/Local Storage Ord_detail Storage Ord_detail _archive table Txn_detail Txn_detail (cold rows only) Compute Regular backup & restore Table 2Table 1 Table 1’ Shallow backup & restore Table 2Table 1 Table 2 Table 1’ On-premises
- 47. Backup Space Consumption Difference
- 50. StretchDB – automatic scaling Azure On- premises SQL Server instanceOn-premises application(s) DB in SAN/Local Storage Ord_detail Storage Shard 1 Ord_detail _archive table Txn_detail Txn_detail (cold rows only) Compute Storage Shard 2 Ord_detail_a rchive table Txn_detail (cold rows only) Transparent scaleout On-premises
- 51. You can scale your service by adjusting the DSU
- 52. SELECT * FROM Department FOR SYSTEM_TIME AS OF '2010.01.01' Facts: 1. History is much bigger than actual data 2. Retained between 3 and 10 years 3. “Warm”: up to a few weeks/months 4. “Cold”: rarely queried Solution: History as a stretch table: PeriodEnd < “Now - 6 months” Azure SQL Database
- 65. Pricing (via azure price calculator)
- 74. “ ” Questions? Ask and you may or may not be answered. The presentation will be uploaded on Docs.com Join us @ https://www.facebook.com/groups/phissug Majority of the parts of the presentation were taken from Joe Yong’s presentation of Stretch Db and Microsoft Introduction of SQL Server 2016’s new features at the Microsoft Ignite 2015 The videos of StretchDb and AlwaysEncrypted were taken from Microsoft’s New SQL. No Equal Series @ https://channel9.msdn.com
- 75. References Stretch Database: https://msdn.microsoft.com/en-us/library/dn935011.aspx Stretching On-Premises Databases to the Cloud: https://channel9.msdn.com/events/Ignite/2015/BRK2574 SQL Server Stretch DB: https://channel9.msdn.com/Shows/Data-Exposed/SQL-Server- Stretch-DB Stretch Database Securely and transparently leverage infinite storage and compute capacity in Azure with SQL Server 2016: https://channel9.msdn.com/events/DataDriven/SQLServer2016/StretchDatabase Microsoft SQL Server 2016, Stretch Database: https://channel9.msdn.com/events/DataDriven/New-SQL-No-Equal/Microsoft-SQL- Server-2016-Stretch-Database Introduction to Stretch Database: http://sqlperformance.com/2015/08/sql-server- 2016/intro-stretch-database
- 76. References Stretch Database Pricing: http://sqlinthewild.co.za/index.php/2016/04/19/sql-2016- features-stretch-database/ Stretch Database Pricing2: http://www.sqlservercentral.com/articles/Editorial/142696/ Stretch Database Pricing2 Discussion: http://www.sqlservercentral.com/Forums/Topic1794699-263-1.aspx MS Pricing Stretch Database: https://azure.microsoft.com/en-us/pricing/details/sql- server-stretch-database/ MS Pricing Storage: https://azure.microsoft.com/en-us/pricing/details/storage/ Stretch Database Pros and Cons: http://kevin3nf.blogspot.com/2016/06/sql-server- 2016-stretch-database.html
Editor's Notes
- Kung may students man sa inyo, pwede nyokong kuning resource speaker. I usally discuss things about C#, ASP.NET MVC, LINQ, MS SQL Server, MS Excel, Power BI, Power Query. Certificate at food lang sapat na.
- The curious case of hybrid and cloud.
- Source: https://msdn.microsoft.com/en-us/library/dn935011(v=sql.130).aspx Stretch Database lets you archive your historical data transparently and securely. In SQL Server 2016 Community Technology Preview 2 (CTP2), Stretch Database stores your historical data in the Microsoft Azure cloud. After you enable Stretch Database, it silently migrates your historical data to an Azure SQL Database. You don't have to change existing queries and client apps. You continue to have seamless access to both local and remote data. Your local queries and database operations against current data typically run faster. You typically enjoy reduced cost and complexity.
- We have two conflicting requirements, and we need to find a middle ground between them.
- The proposed solution is SQL Server 2016’s StretchDB
- By enabling stretchDB we are creating a way for the DB Engine to connect to an Azure SQL Database, and do StretchDB Operations there such as migrating rows and querying the Azure SQL Database.
- To enable Stretch on a database. Right click on the desired database. Point to tasks. Point to stretch. Click enable. This will take you to the StretchDB Wizard.
- The wizard takes you to a brief introductory window. Read it if it interests you, after reading click next.
- Select the tables where the data to be migrated to azure is located and then click next. By default SQL Server Migrates the whole table to Azure.
- After selecting the tables, you will be prompted for your Azure Credentials. Sign in to your azure account and then proceed to the next steps.
- After signing in you need to specify the Following: The azure subscription that you’ll gonna use. The region where the server is located, or where the new server will be located if you’ll opt for new server. The name of the server. This is auto generated if you’re gonna create a new server. The server admin credentials. This will create a new server login and a new Azure SQL Database regardless if you created a new SQL Server or not. After specifying, click next.
- Enter password for the database master key. The database master key secures the credentials that Stretch Database uses to connect to the remote database.
- This dialog allows you to create exceptions for Azure’s firewall. At least add an exception to the Source SQL Server Public IP.
- This dialog shows the summary of the setup. It also includes an estimate of the cost of stretching your database.
- When you proceed to stretching you’ll have to wait until SQL Server is done with its checks, and until Azure is done with the provisioning.
- After everything’s done, the Icon of your database will be changed into something like that one above.
- In case that you need to enable stretching to another table on the same database. Just right click the table, point to stretch and then click enable. You will be shown a dialog similar to what you have seen before.
- This dialog shows the same introduction to stretchDB
- This is similar to the previous dialog where you selected the tables that you want to stretch. However this time, only one table is shown because you specifically chosen the table from the object explorer.
- This shows a summary of the operations that will be carried out.
- After confirming the stretch, the wizard will show you thee status of stretching, you have to wait until SQL Server finishes its operations. After SQL Server is finished, the table is as good as stretched. Please note that Migration does not begin immediately, specially if a lot of this is happening on the server.
- In case that the cold data are mixed with the hot data on a single table, you can specify a criteria for migration by clicking on the cell under Migrate column of the corresponding row of the table.
- In this dialog you can specify the condition for migration. This dialog only allows simple conditions (one column only).
- In case that you need more complex condition, you can create a function as formatted above and create a where clause. This where clause tells if the row is eligible for migration or not. This msdn documentation discusses more about the limitations regarding the complexity of the where clause: https://msdn.microsoft.com/en-us/library/mt613432.aspx
- The way StretchDB migrates data to Azure is more of clump by clump. And it usually depends on fast is your upload speed. During the demonstration on Microsoft Ph, StretchDb was migrating in 100k rows per transaction, when I made this presentation at our house, the migration is 10k rows per transaction. The frequency of these transactions depends if the server is busy or not. SQL Server handles this functions.
- To monitor the status of StretchDb and its migration: Right click on the database. Point to tasks. Point to stretch. Click Monitor A dashboard will show up on SSMS.
- This dashboards shows an overview of the stretching, which include the current size of the database (Data + Log), the Azure Details, and the tables that are configured for stretch.
- You will find out that the tables that are configured for stretch shows the Rows on the table, the rows that are currently on the local server, and the rows that has been migrated to azure. You can also click on the “View Stretch Database Health Events”, this will show a table of events related to Stretch DB.
- The table of events shows the name of the event and the time when it happened. And as you can guess it the “stretch_table_row_migration” is the event where the DB Engine is migrating rows from the local database to Azure. It does not happen exactly after we configured stretch DB. In this case it took my SQL Server a minute or two after I’ve successfully configured the table for stretching. And as you can the row migration happens on an average of twice every minute for an idle server.
- This network graph shows the burst of uploads and downloads whenever the SQL Server is migrating rows.
- Other than the monitor, you can also see the migration status by using the dmv sys.dm_db_rda_migration_status. This DMV features a more detailed view of the migration events.
- You can also monitor the space being used by your database using the sp_spaceused stored procedure. The screenshot above shots three different uses of the stored procedure. The first shows the space used by the whole database, the second shows the space used by the local data, and the third shows the space used by the data on Azure. As you can see, as we let StretchDb migrate data from local to Azure, we are freeing up-space on our local servers.
- Comparing StretchDB to a manually archived table: In manually archived table, depending on your implementation, it might require additional SQL Logic to include archived data. On the other hand, on StretchDB querying a stretched table is the same as querying a normal table. The smart query processing automatically include the data that resides on Azure in your query results. It also detects if the query requires data from Azure or not.
- In this screenshot StretchDemoSimpleFilter table migrates data that has ID values greater than 10000. So when we need those data that has ID Column below 10000, the query execution plan shows that we are querying normally.
- On the other hand, if we issued a query that requires data from Azure, we can see a different execution plan. A plan which includes a remote query action.
- When you stretched a Child table, the smart query processing is smart enough to know that some of the Child table’s data is in azure during a join.
- However the smart query processor is not quite smart when processing joins, it doesn’t really know if we only need the data that is on the local server or not.
- But when you use the where clause as a guide, the smart query processing gets smarter.
- When you are doing backup and restore, the the DB Engine knows that it will only backup the data that is on the local server. So It doesn’t download all data from Azure just for the backup. Same with Restore.
- Here we can see a Backup operation being performed on a database that doesn’t have any rows migrated to azure.
- Notice the difference between the time of backup after we have migrated a lot of rows to azure
- We can also notice the difference between the size of the database backup file.
- Here we can see a restore operation being performed on a database that doesn’t have any rows migrated to Azure.
- I was expecting that it would yield to a faster restore but weird things happened when I attempted to do a restore. My restore database speed won’t get back to 29MB/Sec
- One of the neat feature of Cloud is agility on scaling. Instead of waiting for your order of hardware, you’ll just a few clicks here and the additional load will be address without further effort needed.
- This scaling feature of Azure is also available in the StretchDB Service. Just adjust the amount of DSU that you want. DSU is database stretch unit. It is a unit of measure created by Microsoft so that Azure subscribers would have a relative measure to compare StretchD Service.
- Source: http://channel9.msdn.com/Shows/Data-Exposed/Temporal-in-SQL-Server-2016 Example of using temporal tables with Azure SQL Database stretch tables.
- I wasn’t able to complete my screenshots because I ran out of azure credits. If you don’t mind experimenting, you can create a System-Versioned table, and stretch the history table. Do changes, and see if smart query processing will be applied on the For Clause. During out demonstration, the For clause doesn’t seem to exhibit that behavior.
- You cannot stretch the System-Versioned Table, but you can stretch the History Table.
- A normal query on the table will not hit the Azure Database because the temporal table itself is not stretched.
- This shows that the smart query processor is taking into account the queries for temporal tables. However, it is not behaving as I expect it to be. The values of the rows are not shown as of the specified date. And when you try to disable the stretch database on the history table SSMS is showing null exception errors.
- SQL Server Always Encrypted. Whend ata is being migrated from Local Database to Azure Database, the data is encrypted before it is being sent to azure. But if that security is not enough, you can enable Always Encrypted on a selected column and proceed with the migration.
- To encrypt columns, right click the database and click encrypt columns.
- You will be taken into a wizard that shows an overview of the always encrypted feature.
- Choose which columns are to be encrypted and select the encryption type that we want to.
- Configure master key and then press next.
- This part of the wizard warns you about the resources that the Always Encrypted will eat up.
- This shows the summary of the tasks that will be done.
- After Sql server database engine is finished with the encryption, you can proceed to stretching of the table as normal.
- Configure stretch database on the encrypted table.
- After configuring and verifying if it has been stretch, we take a loot at our encrypted table.
- As you can see even if the columns are encrypted, they are still stored on the Microsoft azure and they are still encrypted and still included in the query.
- This shows that the query execution plan for a stretched table is the same as that of a an encrypted table.
- The pricing calculator is located here: https://azure.microsoft.com/en-us/pricing/calculator/ it gives you a good estimate on how much will it cost for implementing the stretchDB feature.
- To pause the Stretch feature means temporarily stopping the Migration of rows from local to azure. To pause it, right click on the table, point to stretch, and Click Pause.
- Clicking on pause will show you a dialog that looks the same as above, wait until the operations has succeeded and then click close.
- If you want to stop stretching, you can disable it by right clicking on the stretched table, point to stretch, point to disable and click on any of the two choices. You have two choices: Bring back the data from azure or Leave the data in azure.
- If you’ve chosen to bring back the data, SQL Server will begin trickling the Azure SQL Database. It will move clump by clump all of the data that were stored on azure.
- Once that the database’s stretch has been disabled. Data left in azure will no longer be queried smartly.
- It only shows weird execution plan when we try to hit the remote database using our queries.
- In the event that you ran out of Azure credits while a table is being stretched. You will not be able to run any queries pertaining to the stretched table. Regarding the question: “Can I bring back my data if I ran out of azure credits?”, I cannot answer this in plain manner. All I can say is that I cannot get my data back (using the disable + bring back data from azure option) after I ran out of credits. The StretchDB Monitor says: login account is not a valid azure account.
- Although a few days after running out of credits, I happen to see this green graph on my chart. I’m pretty sure it won’t show up as a charge on my credit card :P