SESSION CODE: COS311<br />Migrating Applications to SQL Azure<br />Lynn Langit<br />Sr. Developer Evangelist<br />Microsof...
Migrating Applications to SQL Azure<br />Overview<br />SQL Azure<br />App Migration<br />Data Access Tier Considerations<b...
Developer Experience<br />Use existing skills and tools<br />The Windows Azure Platform<br />Relational data<br />Manageme...
SQL Azure DatabaseThe first and only true relational database as a service<br />Subtitle color<br />Elastic Scale<br />Sel...
Leverage existing skills through existing ecosystem of developer and management tools
Explore new data application patterns
Database utility; pay as you grow
Flexible load balancing
Business-ready SLAs
Enable multi-tenant solutions
World-wide presence
Easy provisioning and deployment
Auto high-availability and fault tolerance
Self-maintaining infrastructure; self-healing
No need for server or VM administration</li></li></ul><li>Application Topologies<br />SQL Server<br />App code/ Tools<br /...
When Migrating Applications<br />Data Access APIs Support<br />Security Model<br />Connections  & Sizing<br />Admin and Pr...
Data Access APIs<br />Supported APIs<br />Connection String<br />ADO.Net- Encrypt=True and add @servername to User ID<br /...
Data Access APIs cont.<br />Protocols<br />Authentication Mode<br />
Connectivity & Sizing<br />Robust Connection Management through Retry Logic<br />More On this Topic - COS305: Microsoft SQ...
Authentication- SQL Server (only)<br />Manage Logins<br />CREATE LOGIN [testlogin] WITH PASSWORD='password‘<br />Manage Us...
Firewall Management<br />Portal Access<br />Programmatic Access<br />System Views and Stored Procedures - here<br />View: ...
Collations<br />Default Collation<br />Set to SQL_LATIN1_GENERAL_CP1_CI_AS<br />Applies to…<br />Metadata, TempDb and Defa...
Feature Parity<br />Admin Surface – no physical administration<br />You have a master database <br />You do NOT have serve...
Migrating Applications<br />
Database Migration<br />Moving Schema and / or Data<br />Source Database types<br />SQL Server<br />MySQL<br />Access<br /...
From SQL Server to SQL Azure<br />Tool Choices<br />SSMS for R2 > Generate Script Wizard<br />SSMS for R2 > Import / Expor...
SSMS– Generate  Script Wizard<br />Migration of schema and/or data w/ fine grain control<br />Pros - Native support for SQ...
Generate Script Wizard - DEMO<br />Migrating Databases to SQL Azure<br />
SQL Azure Migration Wizard<br />http://sqlazuremw.codeplex.com<br />
Migration Assist for MySql/ Access (CTP)<br />Scenario - Auto porting of schema, database code and data from MySql and Acc...
MySQL to SQL Azure - DEMO<br />SQL Server Migration Assistant <br />Version for MySQL<br />Version for Access<br />
MySQL Migration Tool<br />
DAC (Data-tier Application) Package<br />DACPac= Single unit for authoring, deploying, and managing data-tier objects (sch...
DACs - DEMO<br />Migrating Databases to SQL Azure<br />
Transferring Large Data to SQL Azure<br />Best Practices for efficient Data movement to SQL Azure<br />Use the right tools...
BCP & Bulk Copy APIs for Large Data Transfer<br />Optimize Databases for Data Import<br />Disable/Delay creation of none C...
Import/Export Wizard – for data transfer<br />Scenario- Simplified wizard for migrating schema and data through SSIS<br />...
SSIS  Data Flow Task (Large Data Xfer)<br />Diverse Source and destinations - To/From Flat Files, ADO.Net, OleDB<br />Full...
SSIS package - DEMO<br />Use supported features only, such as bcp, but not BULK INSERT, etc…<br />
Parallelization & Network Perf<br />Latency & Bandwidth for high performance import & export<br />Latency is about laws of...
Datacenter Locations<br />
BCP Performance– test set up example<br />Data Distribution:<br />Row Size: 600Bytes<br />Row Count: 100K<br />Pages: ~770...
BCP Upload Perf- Rows/Sec<br />
BCP Download Perf– Rows/sec<br />
Upcoming SlideShare
Loading in …5
×

Migrating Data to SQL Azure

3,516 views

Published on

deck from TechEd Africa Oct 2010

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,516
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
50
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Code Far works with any type of code – i.e. Java, etc.. Does NOT have to be Azure or .NET
  • For MySQL - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=69739c8c-ac82-41de-b9e6-8fa5ae2594d9&amp;displaylang=enFor Access - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=5abe098d-c7e1-46c6-994a-09a2856eef0b&amp;displaylang=en
  • SELECT SUM(reserved_page_count) * 8192 FROM sys.dm_db_partition_statshttp://www.microsoft.com/windowsazure/sqlazure/datasync/default.aspxhttp://blogs.msdn.com/b/sync/archive/2010/08/31/sql-server-to-sql-azure-synchronization-using-sync-framework-2-1.aspxSupplied DMVsSupported Dynamic Management Viewssys.dm_exec_requestssys.dm_exec_text_query_plansys.dm_exec_sessionssys.dm_exec_connectionssys.dm_tran_database_transactionssys.dm_db_partition_statssys.dm_exec_query_plansys.dm_exec_query_statssys.dm_tran_active_transactionssys.dm_tran_session_transactionssys.dm_exec_sql_textsys.dm_tran_locksMonitoring SQL Azure using DMVhttp://msdn.microsoft.com/en-us/library/ff394114.aspx
  • Migrating Data to SQL Azure

    1. 1. SESSION CODE: COS311<br />Migrating Applications to SQL Azure<br />Lynn Langit<br />Sr. Developer Evangelist<br />Microsoft Corporation<br />Blogs.msdn.com/SoCalDevGal<br />v1.1<br />
    2. 2. Migrating Applications to SQL Azure<br />Overview<br />SQL Azure<br />App Migration<br />Data Access Tier Considerations<br />Database Migration<br />Migrating to SQL Azure from MySql<br />Moving Schema and Data into SQL Azure<br />Transferring Large Data into SQL Azure<br />Future<br />
    3. 3. Developer Experience<br />Use existing skills and tools<br />The Windows Azure Platform<br />Relational data<br />Management<br />platform<br />Compute<br />Storage<br />Management<br />AppFabric<br />Flexible APIs<br />Connectivity<br />Access control<br />Information Marketplace<br />Reporting & BI<br />Billing & Payments<br />
    4. 4. SQL Azure DatabaseThe first and only true relational database as a service<br />Subtitle color<br />Elastic Scale<br />Self-managed<br />Developer Agility<br /><ul><li>Build cloud-based database solutions on consistent relational model
    5. 5. Leverage existing skills through existing ecosystem of developer and management tools
    6. 6. Explore new data application patterns
    7. 7. Database utility; pay as you grow
    8. 8. Flexible load balancing
    9. 9. Business-ready SLAs
    10. 10. Enable multi-tenant solutions
    11. 11. World-wide presence
    12. 12. Easy provisioning and deployment
    13. 13. Auto high-availability and fault tolerance
    14. 14. Self-maintaining infrastructure; self-healing
    15. 15. No need for server or VM administration</li></li></ul><li>Application Topologies<br />SQL Server<br />App code/ Tools<br />App code/ Tools<br />Application/ Browser<br />SQL Azure / Data Sync<br />Windows<br />Azure<br />Windows<br />Azure<br />App Code<br />(ASP.NET)<br />App Code<br />(ASP.NET)<br />T-SQL / TDS<br />App Code<br />(ASP.NET)<br />App Code<br />(ASP.NET)<br />TSQLTDS<br />TSQLTDS<br />SQL Azure <br />SQL Azure <br />SQL Azure <br />Code Far<br />Code Near<br />Hybrid<br />
    16. 16. When Migrating Applications<br />Data Access APIs Support<br />Security Model<br />Connections & Sizing<br />Admin and Programmability Surface Support<br />Collation Behavior<br />
    17. 17. Data Access APIs<br />Supported APIs<br />Connection String<br />ADO.Net- Encrypt=True and add @servername to User ID<br />ODBC - Encrypt=yes and add @servername to Uid<br />
    18. 18. Data Access APIs cont.<br />Protocols<br />Authentication Mode<br />
    19. 19. Connectivity & Sizing<br />Robust Connection Management through Retry Logic<br />More On this Topic - COS305: Microsoft SQL Azure Development Best Practices<br />Failover<br />HW Failures or Upgrades<br />Resource Management Operations<br />Throttling and Load Balancing<br />ALTER DATABASE (T-SQL)<br />Changes to Database Name <br />Changes to MAXSIZE/EDITION<br /> ALTER DATABASE [TestDB] MODIFY (MAXSIZE=50GB)<br /> GO<br />
    20. 20. Authentication- SQL Server (only)<br />Manage Logins<br />CREATE LOGIN [testlogin] WITH PASSWORD='password‘<br />Manage User<br />CREATE USER testuser FROM LOGIN testlogin;<br />Manage Server Roles<br />EXEC sp_addrolemember ‘dbmanager', 'testuser'; [securityadmin role]<br />EXEC sp_addrolemember ‘loginmanager', 'testuser'; [dbcreator]<br />Granting Database Access to a Login<br />Create Login and Database in Master<br />Create User and Grant Role in User Database<br />Views<br />sys.sql_logins.sys.databases, sys.database_principal<br />Security Management<br />
    21. 21. Firewall Management<br />Portal Access<br />Programmatic Access<br />System Views and Stored Procedures - here<br />View: sys.firewall_rules<br />Stored Procedure:sp_set_firewall_rule<br />Stored Procedure:sp_delete_firewall_rule<br />API Implementation Example- here<br />
    22. 22. Collations<br />Default Collation<br />Set to SQL_LATIN1_GENERAL_CP1_CI_AS<br />Applies to…<br />Metadata, TempDb and Default for User Data <br />Object Names (tables, sprocsetc), String variables and Columns when no collation is specified. <br />Use column and/or expression level collation when needed<br />CREATE TABLE [dbo].[ErrorLog](<br /> [ErrorLogID] [int] IDENTITY(1,1) NOT NULL,<br /> [ErrorTime] [datetime] NOT NULL,<br /> [UserName] [nvarchar(4000)] COLLATE Turkish_100_CI_AI NOT NULL,<br /> [ErrorNumber] [int] NOT NULL,<br /> [ErrorMessage] [nvarchar](4000) COLLATE Japanese_CI_AS NOT NULL,<br /> CONSTRAINT [PK_ErrorLog_ErrorLogID] PRIMARY KEY CLUSTERED ([ErrorLogID] ASC)<br /> )<br />declare @strnvarchar(128) COLLATE Greek_CS_AI<br />
    23. 23. Feature Parity<br />Admin Surface – no physical administration<br />You have a master database <br />You do NOT have server level constructs (sp_configure, endpoints, DBCC commands, server level DMVs and System Views…)<br />Programmability Surface<br />Most RDMS features available<br />Partially available - USE, XML processing, deprecated T-SQL etc. – list here<br />Some Features are NOT available today<br />Full-text Search, Remote Data Access and Linked Servers, Distributed Transactions, Change Tracking, Service Broker etc.<br />List is available here<br />
    24. 24. Migrating Applications<br />
    25. 25. Database Migration<br />Moving Schema and / or Data<br />Source Database types<br />SQL Server<br />MySQL<br />Access<br />Other RDMS<br />
    26. 26. From SQL Server to SQL Azure<br />Tool Choices<br />SSMS for R2 > Generate Script Wizard<br />SSMS for R2 > Import / Export Wizard<br />Copy Database command<br />SSIS for R2<br />SQL Azure Migration Wizard<br />3rd Party<br />
    27. 27. SSMS– Generate Script Wizard<br />Migration of schema and/or data w/ fine grain control<br />Pros - Native support for SQL Azure Schema options: “Engine Type = SQL Azure”<br />Ensures correct options and settings are applied for the TSQL script generation.<br />Requires explicit action on unsupported objects. <br />Cons- Verbose – INSERT Statements instead of raw data<br />Data is scripted with a fixed 100 row batch size.<br />Edit “GO” statements between small batches<br />Round-trip Efficiency<br />Use “SET NOCOUNT ON”<br />
    28. 28. Generate Script Wizard - DEMO<br />Migrating Databases to SQL Azure<br />
    29. 29. SQL Azure Migration Wizard<br />http://sqlazuremw.codeplex.com<br />
    30. 30. Migration Assist for MySql/ Access (CTP)<br />Scenario - Auto porting of schema, database code and data from MySql and Access to SQL Azure<br />SQL Server Migration Assistant for MySql and Access<br />Supports MySQL 4.1 and up<br />Support Access v 97 and up<br />SQL Server versions supported (all editions)<br />SQL Azure, SQL Server 2005, SQL Server 2008 and 2008 R2<br />
    31. 31. MySQL to SQL Azure - DEMO<br />SQL Server Migration Assistant <br />Version for MySQL<br />Version for Access<br />
    32. 32. MySQL Migration Tool<br />
    33. 33. DAC (Data-tier Application) Package<br />DACPac= Single unit for authoring, deploying, and managing data-tier objects (schema)<br />Development Lifecycle (VS 2010)<br />Creating / Editing DACs<br />Schema and DB Code Development<br />Code Analyses, Deployment Policy Settings<br />Schema Comparison and more…<br />Management Lifecycle (SSMS 2008 R2)<br />Extracting DACs (as files)<br />Managing DACs (deploy or upgrade)<br />Registering existing database as DACs<br />If ‘regular’ into MSDB<br />If ‘Azure’ into master<br />If using UCP into UCP database<br />
    34. 34. DACs - DEMO<br />Migrating Databases to SQL Azure<br />
    35. 35. Transferring Large Data to SQL Azure<br />Best Practices for efficient Data movement to SQL Azure<br />Use the right tools is key <br />BCP<br />SSIS <br />Network performance considerations<br />Latency vs Bandwidth<br />
    36. 36. BCP & Bulk Copy APIs for Large Data Transfer<br />Optimize Databases for Data Import<br />Disable/Delay creation of none Clustered Indexes<br />Order Data on the Clustered Index<br />Disable Triggers and Constraint checking<br />-N Native mode so no data type conversion needed. <br />-c Character mode for interoperability<br />-b batchsize will let you control transaction size<br />-h”TABLOCK, ORDER(…)” optimize writes<br />
    37. 37. Import/Export Wizard – for data transfer<br />Scenario- Simplified wizard for migrating schema and data through SSIS<br />Pros<br />Great performance out of the box<br />Allows ‘save as package’ for full control<br />
    38. 38. SSIS Data Flow Task (Large Data Xfer)<br />Diverse Source and destinations - To/From Flat Files, ADO.Net, OleDB<br />Fully programmable flow <br />Loops, Sorts, Conditional operators, XML/WebServices Processing etc<br />VS Debugging support with data viewers, watches and conditional breakpoints<br />Optimize Databases for Data Import<br />Disable/Delay creation of none Clustered Indexes<br />Order Data on the Clustered Index<br />Disable Triggers and Constraint checking<br />Batch Size: Adjust the transaction size<br />Buffer and Blob temp storage area; spread over to multiple drives for large data transfers<br />Parallelization – based on execution trees, task will auto parallelize<br />
    39. 39. SSIS package - DEMO<br />Use supported features only, such as bcp, but not BULK INSERT, etc…<br />
    40. 40. Parallelization & Network Perf<br />Latency & Bandwidth for high performance import & export<br />Latency is about laws of physics<br />Long distances means high latency<br />Parallelization can help mitigate some latency<br />Bandwidth is key! <br />Once you reach the bandwidth limit parallelization won’t help<br />See also COS13-INT: Database Performance in a Multi-tenant Environment<br />
    41. 41. Datacenter Locations<br />
    42. 42. BCP Performance– test set up example<br />Data Distribution:<br />Row Size: 600Bytes<br />Row Count: 100K<br />Pages: ~7700<br />Total Size: ~60MB<br />Even Distribution on Pkey<br />Constraints and Indexes<br />Compound Pkey (int+int) <br />None-clustered key on char(50)<br />No triggers or relationships<br />CREATE TABLE [dbo].[table1](<br />[c01] [int] NOT NULL,<br />[c02] [int] NOT NULL,<br />[c03] [smallint] NOT NULL,<br />[c04] [int] NOT NULL,<br />[c05] [smallint] NOT NULL,<br />[c06] [smallint] NOT NULL,<br />[c07] [char](50) NOT NULL,<br />[c08] [char](24) NOT NULL,<br />…<br />[c17] [char](24) NOT NULL,<br />PRIMARY KEY CLUSTERED <br />([c01] ASC,[c02] ASC))<br />CREATE NONCLUSTERED INDEX [index1] <br />ON [dbo].[table1] ([c07] ASC)<br />
    43. 43. BCP Upload Perf- Rows/Sec<br />
    44. 44. BCP Download Perf– Rows/sec<br />
    45. 45. Guidance on Data Transfer to SQL Azure<br />Split data transfer into multiple concurrent streams<br />Tune BCP batch size and network packet size option <br />Add non clustered indexes after loading data to SQL Azure. <br />Two additional indexes created before loading the data increased the final database size by ~50% and increased the time to load the same data by ~170%.<br />If, while building large indexes, you see a throttling-related error message, retry using the online option.<br />
    46. 46. Monitoring Tasks<br /><ul><li>Space Used
    47. 47. Monitor space used via ‘sys.dm_db_partition_stats’
    48. 48. Trap ‘out of space exception’ – SQL Error 40544
    49. 49. Synchronized Copies
    50. 50. Database copy
    51. 51. SQL Azure Synchronization Provider
    52. 52. Other monitoring tasks – use supplied DMVs (list in notes for this slide</li></li></ul><li>Database Migration Tools <br />
    53. 53. Future for Migration <br />More information on all upcoming features at<br />DAT209: What’s New in Microsoft SQL Azure<br />Migrating Applications<br />Connectivity<br />OleDb Support<br />SQL Server Parity<br />DB Collation Support<br />OData Support – SQL Azure Labs<br />Further reach for SQL Azure data<br />
    54. 54. Related Content<br />Required Slide<br />Speakers, please list the Breakout Sessions, Interactive Sessions, Labs and Demo Stations that are related to your session.<br />Breakout Sessions<br />AZP301 – A Lap Around the Azure Platform<br />AZP302 – Using Visual Studio 2010 to Build Applications that run on Azure<br />DAT207 – What’s New in Microsoft SQL Azure<br />AZP303 – Migrating Applications to Microsoft SQL Azure<br />
    55. 55. Track Resources<br />Team Blog – http://blogs.msdn.com/sqlazure<br />MSDN Dev Center – http://msdn.microsoft.com/azure/sqlazure<br />SQL Azure Labs – http://www.sqlazurelabs.com<br />Follow us on Twitter - @sqlazure<br />Windows Azure Platform Training Kit - http://bit.ly/dgvMv0<br />
    56. 56. Resources<br />Required Slide<br />Learning<br />Sessions On-Demand & Community<br />Microsoft Certification & Training Resources<br />www.microsoft.com/teched<br />www.microsoft.com/learning<br />Resources for IT Professionals<br />Resources for Developers<br />http://microsoft.com/technet<br />http://microsoft.com/msdn<br />
    57. 57. Complete an evaluation via CommNet and Tag to win amazing prizes!<br />
    58. 58. TeachingKidsProgramming.org<br /><ul><li>Do a Recipe  Teach a kid
    59. 59. Free Courseware (recipes)
    60. 60. Ages 10 ++
    61. 61. Microsoft SmallBasic</li></li></ul><li>© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.<br />The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.<br />

    ×