SQL Azure March 2010


Published on

deck from my presentation at 'Day of Azure' in San Diego in March 2010

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Steve Hale (Microsoft) authored this article at http://blogs.msdn.com/sqlnativeclient/archive/2010/02/12/using-sql-server-client-apis-with-sql-azure-vversion-1-0.aspxAlso updated MSDN Dev Guide 'How to connect' http://msdn.microsoft.com/en-us/library/ee621787.aspxUsing SQL Server Client APIs with SQL Azure Version 1.0 This post describes some of the issues that the application developer should be aware of when writing ODBC, ADO.NET SQLClient, BCP, and PHP code that connects to SQL Azure. See also SQL Azure documentation on MSDN (http://msdn.microsoft.com/en-us/library/ee336279.aspx). Supported components1. Microsoft SQL Server Native Client 10.0 ODBC Driver or newer. This is a free download.2. Microsoft PHP 1.1 driver or newer. Again a free download.3. SQL Server 2008 BCP.EXE or newer. The caution here is that the queryout option is not fully supported for SQL Server 2008 RTM. Queryout is expected to be fully supported in a future release of SQL Server.4. ADO.NET 3.5 SqlClient or newer.Any other components are NOT currently supported for connecting to SQL Azure. In particular it should be noted that OLEDB connections are not currently supported from any OLEDB provider, even SQL Server 2008 Native Client OLEDB provider. ODBC connections using the on-the-box ODBC driver “SQL Server” are not currently supported.Connecting1. valid server name, user ID, and password for SQL Azure2. On the SQL Azure portal, set the firewall rules to allow your client IP address to connect to SQL Azure. Additionally, in some corporate environments, you might need to install Microsoft Firewall Client for ISA Server on the client machines or connections will not succeed. And in those environments, using IIS might require the default IIS settings for Application Pool Identity to be changed. Here is how that setting is changed: Open IIS Manager in Admin Tools, select Application Pools, Pick the pool, Advanced Setting and under Process Model change Identity from BuiltInAccountApplicationPoolIdentity to Custom account domain account. Restart IIS.SQL Azure supports only TCP/IP connections using port 1433 (default for most SQL Server APIs) if other protocol choices (named pipes or using other ports) Additionally, most client APIs do not send the server name when connecting, thus the server name should be included as part of the user ID. Currently the only exception to this is .NET 4.0 which does send the server name. ODBC, BCP, PHP, and .NET 3.5 all require the server name to be sent as part of the user ID. This may change in a future release. Connection strings are all similar for each API stack and require the server name, user ID, password, and optionally the database. Trusted connections (also known as Integrated Security or Windows Authentication) are not supported. MARS is not yet supported by SQL Azure so make sure MARS is not enabled in the connection string.Creating and Connecting to a DatabaseSQLCmd example: sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword>Note the required server name (in yellow) included as part of the user ID.In the above example the database was not specified. In that case the default database “master” is used. You must connect to the master database to create databases or drop databases. Since the “USE database” command is not supported, you must disconnect after creating a database and reconnect directly to the user database. In this example we connect directly to the Northwind database.sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword> -d Northwind Creating tablesTable creation is restricted to user databases; you cannot create tables in the master database. Table creation is substantially the same as on SQL Server, with the exception that certain data types are not supported, for example CLR User Defined Types, Filestream, and row guid types. Also, it should be mentioned that SQL Azure requires a clustered index or primary key (which uses a clustered index by default) before data can be inserted into a table. For this reason a clustered index or primary key must always be created for any table intended to contain data.Example:Create table employees(EmployeeID int primary key, EmployeeName varchar(30))OrCreate table employees(EmployeeID int, EmployeeName varchar(30))Create clustered index empIndex on employees(EmployeeID) Unsupported features· “USE database” is not supported· 3-part names are not supported unless the database being referenced is the current database.· Creating or dropping a database when not in the master database· Insertion of data into a table without a clustered index.· MARS connections· Trusted connections· CLR UDTs· XML schema collections· Distributed transactions (local transactions are supported)· AttachDB· Certain stored procedures, for example sp_who and undocumented stored procedures used to support OLEDB schema rowsets. Deprecated functionalitySome features are deprecated in SQL Server and should not be used in new application development. In most cases SQL Azure does not support deprecated SQL Server features. Some examples of these are:· System tables such as sysobjects, sysusers, or sysdatabases. Use the new system views instead. See http://msdn.microsoft.com/en-us/library/ms187997.aspx for a mapping to the new system views.· Numbered stored procedures· “select into” syntax· “COMPUTE BY” clause· Two-part drop index statements See http://msdn.microsoft.com/en-us/library/ee336281.aspx for more detailed Transact-SQL support.Connection behaviorSQL Azure connections will behave somewhat differently than SQL Server connections. They can be dropped for a number of reasons:· Excessive resource usage· Long-running queries· Long-running single transactions· Idle connections· Failover due to server failuresFor this reason it’s best to avoid caching connections for later use. A better strategy is to open the connection when needed, use it, and then close it. Additionally, it is recommended that applications provide retry/reconnect logic to recover from dropped connections.LatencyLatency may be a concern when using SQL Azure unless the SQL Azure server is located near the application (“code near” scenario), such as with Windows Azure and a co-located SQL Azure. Making many round trips to the server can introduce noticeable delay in an application that previously performed adequately when running against a SQL Server on the intranet. Ensure the application connects to the nearest SQL Azure server and that round trips are minimized. SUMMARYü Use only supported client components to connectü Connect using TCP/IP and port 1433ü Use SQL Authentication only, not Trusted Connections/Integrated Authentication/Windows Authentication.ü Specify the server name in the user ID.ü Ensure the firewall rules in the SQL Azure portal permit your IP address.ü Install Microsoft Firewall Client for ISA Server if needed.ü Change the IIS Application Pool Identity if needed.ü Make sure MARS is not enabled.ü Connect to the master database to create or drop databasesü Create a user database for your application since tables cannot be created in masterü Disconnect and reconnect to change databasesü Create a clustered index or primary key on a table before inserting data.ü Remove any unsupported or deprecated features from the application.ü Avoid caching connections. Open the connection, use it, and then close it.ü Avoid making numerous round trips to the server.
  • Fan out queries: where a query can be run against every container in parallel and the results returned and a union-all performed at the client.  This is an important scenario for us and we are looking at how we can make this pattern run effiAuto partitioning. Multi-spindle storage solution. 10 drives look like 1 logical drive.
  • Clustered indexes sort and store the data rows in the table based on their key values. There can only be one clustered index per table, because the data rows themselves can only be sorted in one order. For more information about clustered index architectureWith few exceptions, every table should have a clustered index defined on the column, or columns, that offer the following: Can be used for frequently used queries. Provide a high degree of uniqueness. Can be used in range queries.Distributed Query – reference multiple linked servers and can perform either updae or read operations against each individual linked serverDistributed transactions span two or more servers known as resource managers. The management of the transaction must be coordinated between the resource managers by a server component called a transaction manager.DDL – data definition language statement to create drop or alter mining models and multidimensional objects such as cubes and dimensions
  • Database files can be grouped together in filegroups for allocation and administration purposes. Some systems can improve their performance by controlling the placement of data and indexes onto specific disk drives. Filegroups can aid this process. The system administrator can create filegroups for each disk drive, then assign specific tables, indexes, or the text, ntext, or image data from a table, to specific filegroups.
  • We have included this feature comparison table in anticipation of your likely questions about differences between using a relational database table as you may be currently doing with your SQL Server databases and the new Windows Azure Tables included in Windows Azure.
  • · sys.dm_exec_connections – This view returns information about the connections established to your database.· sys.dm_exec_requests – This view returns information about each request that executes within your database · sys.dm_exec_sessions – This view shows information about all active user connections and internal tasks. · sys.dm_tran_database_transactions – This view returns information about transactions at the database level.· sys.dm_tran_active_transactions – This view returns information about transactions for your current logical database.· sys.dm_db_partition_stats – This view returns page and row-count information for every partition in the current database.
  • SQL Azure March 2010

    1. 1. Lynn Langithttp://blogs.msdn.com/SoCalDevGaltwitter - @llangit<br />
    2. 2. Windows Azure Platform<br />Compute: Virtualized compute environment based on Windows Server<br />Storage: Durable, scalable, & available storage<br />Management: Automated, model-driven management<br />Database:Relational processing for structured/unstructured data<br />Service Bus: General purpose application bus<br />Access Control: Rules-driven, claims-based access control<br />
    3. 3. Familiar SQL Server relational model<br />Uses existing APIs & tools<br />Built for the Cloud with availability and scale<br />Accessible to all from .NET, PHP, Ruby, or Java<br />Microsoft SQL Azure<br />Clear Feedback: “I want a database in the Cloud”<br />Focus on combining the best features of SQL Server running at scale with low friction<br />
    4. 4. Extending SQL Data Platform to Cloud<br />Reference Data<br />Business Intelligence<br />Data Sync<br />Reporting<br />SQL Azure Database<br />Symmetric Programming Model<br />Data Hub Aggregation<br />Initial Services<br />Database – Core SQL Server database capabilities <br />Future Services<br />Data Sync – Enables the sync framework (soon after PDC)<br />Additional SQL Server capabilities available as a service: Business Intelligence and Reporting<br />New services: Reference Data and Secure Data Hub<br />
    5. 5. Database Choices<br />Value Props:<br />Full h/w control – size/scale<br />100% compatibility<br />Roll-your-own HA/DR/scale<br />Value Props:<br />100% of API surface area<br />Roll-your-own HA/DR/scale<br />On-premise<br /> SQL Server or other s/w on-premise<br /> Resource governance @ machine<br /> Security @ DB Server/OS<br />Value Props:<br />Auto HA, Fault-Tolerance<br />Friction-free scale<br />Self-provisioning<br />High compatibility<br />Dedicated<br />Hosted<br /> Hosted SQL Server or other<br /> Resource governance @ VM<br /> Security @ DB Server/OS<br />SQL Azure (RDBMS)<br />Resources<br />Virtual DB server<br />Resource governance @ LDB<br /> Security @ LDB<br />Shared<br />Low<br />“Friction”/Control<br />High<br />
    6. 6. V1 Application Topologies<br />SQL Azure access from outside MS Datacenter <br />SQL Azure access from within MS Datacenter <br />Application/ <br />Browser<br />App Code / Tools<br />ADO.NET Data Svcs/REST - EF<br />HTTP/S<br />SOAP/REST<br />HTTP/S<br />App Code<br />(ASP.NET)<br />Windows Azure<br />T-SQL (TDS)<br />T-SQL (TDS)<br />SQL Data Services<br />SQL Data Services<br />MS<br />Datacenter<br />MS<br />Datacenter<br />Code Far<br />Code Near<br />
    7. 7. Departmental Applications<br />Web Applications<br />ISV<br />Data Hub (Shortly After V1)<br />Scenarios for V1<br />
    8. 8. “It meets the need of Small Business customers that need a true RDBMS for mission critical applications. After many years of implementing enterprise RDBMS applications for customers with in-house infrastructure and talent, how can I in good conscience develop and implement mission critical applications for my small business customers and leave the OS and RDBMS maintenance to a company with no technical staff?”<br />“Because of the market segment that it addresses, there is no need for enterprise-like functionality, only high-availability and zero-maintenance.”<br />“It is very well priced. You get a true RDBMS for a fraction of typical in-house hosting costs. The fact that it is a shared resource does not diminish the feature-set, it only amplifies the need for a well architected and optimized design.”<br />Customer Quotes<br />
    9. 9. Database Replicas<br />Single Database<br />Multiple Replicas<br />Replica 1<br />Single Primary<br />Replica 2<br />DB<br />Replica 3<br />
    10. 10. SLA Information<br />SQL Azure customers will have connectivity between the database and our Internet gateway. SQL Azure will maintain a “Monthly Availability” of 99.9% during a calendar month. “Monthly Availability Percentage” for a specific customer database is the ratio of the time the database was available to customer to the total time in a month. Time is measured in 5-minute intervals in a 30-day monthly cycle. Availability is always calculated for a full month. An interval is marked as unavailable if the customer’s attempts to connect to a database are rejected by the SQL Azure gateway.Download SQL Azure SLA.<br />
    11. 11. SQL AzureDeployment<br />Web Portal<br />(API)<br />DB Script<br />SQL Azure<br />TDS<br />Support existing and new forms of deployment<br />
    12. 12. Each accounthas zero or more servers<br />Azure wide, provisioned in a common portal<br />Billing instrument<br />Each serverhas one or more databases<br />Contains metadata about the databases and usage<br />Unit of authentication<br />Unit of Geo-location<br />Generated DNS based name<br />Each databasehas standard SQL objects<br />Unit of consistency<br />Unit of multi-tenancy<br />Contains Users, Tables, Views, Indices, etc.<br />Most granular unit of billing<br />Service Provisioning Model<br /> Account<br /> Server<br /> Database<br />
    13. 13. SQL Azure focus on logical administration<br />Schema creation and management<br />Query optimization<br />Security management (Logins, Users, Roles)<br />Service handles physical management<br />Automatically replicated with HA “out of box”<br />Transparent failover in case of failure<br />Load balancing of data to ensure SLA<br />Logical vs. Physical Administration<br />DBA role places more focus on logical management<br />
    14. 14. Uses regular SQL security model<br />Authenticate logins, map to users and roles<br />Authorize users and roles to SQL objects<br />Limited to standard SQL Auth logins<br />Username + password <br />Future AD Federation, WLID, etc as alternate authentication protocols<br />Security Model<br />Security model is 100% compatible with on-premise SQL<br />
    15. 15. SQL Azure portal - http://sql.azure.com<br />Demo – SQL Azure Portal<br />
    16. 16. SQL AzureAccessing databases<br />Web Portal<br />(API)<br />Your App<br />SQL Azure<br />TDS<br />Change Connection String<br />
    17. 17. Use ADO.NET, ODBC, PHP (NOT OLE DB)<br />Client libraries pre-installed in Azure roles<br />Support for ASP.NET controls<br />Clients connect directly to a database<br />Cannot hop across DBs (no USE)<br />May need to include <login>@<server><br />Use familiar tools (sqlcmd, osql, SSMS, etc)<br />SSMS 2008 R2 CTP can connect <br />http://blogs.msdn.com/ssds/archive/2009/11/11/9921041.aspx<br />Connection Model<br />
    18. 18. Connecting to SQL Azure with SSMS 2008 R2 <br />SQL Azure Explorer for VS 2010 http://sqlazureexplorer.codeplex.com/<br />SQLCMD – can also be used<br />SSIS – can also be used<br />Demo – Connect via tools<br />
    19. 19. Programming Model<br />Small Data Sets<br />Use a single database<br />Same model as on premise SQL Server<br />Large Data Sets and/or Massive Throughput<br />Partition data across many databases<br />Use parallel fan-out queries to fetch the data<br />Application code must be partition aware in v1<br />For v1 will publish best practices for scale out<br />Post-v1 we are looking at building an abstraction to hide some of the complexities of partitioning<br />
    20. 20. Maximum single database size is currently 10GB<br />Database size calculation <br />Includes: primary replica data, objects and indexes<br />Does NOT include: logs, master database, system tables, server catalogs or additional replicas<br />V1 does not support auto-partitioning or fan-out queries<br />Must handle partitioning logic within the application<br />Special ConsiderationsDatabase Size<br />
    21. 21. Sharding Databases<br />1 x 10GB database<br />1 Instances<br />10 x 1GB databases<br />10 Instances <br />
    22. 22. Support common application patterns<br />Logical/policy based administration<br />Patterns work from SQL Server to SQL Azure<br />Multi-tenancy considerations<br />Throttling and load balancing<br />Limits on DB size, transaction duration, …<br />Compatibility Goals<br />V1: Address the needs of the majority of web and departmental applications<br />
    23. 23. Compatibility for V1<br />In Scope<br /><ul><li> Tables, indexes and views
    24. 24. w/ clustered indices
    25. 25. Stored Procedures
    26. 26. Triggers
    27. 27. Constraints
    28. 28. Table variables
    29. 29. Session temp tables</li></ul>Out of Scope<br /><ul><li> Distributed Transactions
    30. 30. Distributed Query
    31. 31. CLR
    32. 32. Service Broker
    33. 33. Spatial datatype
    34. 34. Physical server or catalog DDL and views</li></li></ul><li>Constants<br />Constraints<br />Cursors<br />Index management and rebuilding indexes<br />Local temporary tables<br />Reserved keywords<br />Stored procedures<br />Statistics management<br />Transactions<br />Supported T-SQL (v1)<br /><ul><li>Triggers
    35. 35. Tables, joins, and table vars
    36. 36. T-SQL elements
    37. 37. Create/drop databases
    38. 38. Create/alter/drop tables
    39. 39. Create/alter/drop users and logins
    40. 40. Views</li></li></ul><li>CLR<br />Database file placement<br />Database mirroring<br />Distributed queries<br />Distributed transactions<br />Filegroup management<br />Full Text Search<br />Unsupported T-SQL (v1)<br /><ul><li>Global temporary tables
    41. 41. Spatial data and indexes
    42. 42. SQL Server configuration options
    43. 43. SQL Server Service Broker
    44. 44. System tables
    45. 45. Trace Flags</li></li></ul><li>Remove unsupported features, such as…<br />Migrating an existing schema<br /><ul><li>User Defined Data Types
    46. 46. Specifying Filegroups
    47. 47. Extended Properties
    48. 48. USE DATABASE
    50. 50. Various Index options
    51. 51. Windows logins
    52. 52. Unsupported XML features
    53. 53. Indices need to be clustered
    54. 54. Full list at
    55. 55. http://www.microsoft.com/sqlserver/2008/en/us/R2.aspx</li></li></ul><li>Generating, then modifying, the SQL DDL script<br />Loading in the data (bcp, etc…)<br />-OR-<br />SQL Azure Migration Wizard<br />http://sqlazuremw.codeplex.com<br />Demo – Data into the Cloud<br />
    56. 56. Simple Application – just change the connection string!<br />All compatibility requirements must be met (T-SQL) in the DDL<br />Create destination DB schema and populate with data<br />Firewall rules set up via SQL Azure portal (test connectivity)<br />Dem0 – SQL Azure application<br />
    57. 57. Pricing<br />Web Edition<br />Business Edition<br />1 GB Database<br />$9.99 / month<br />Bandwidth<br />$0.10 /GB inbound<br />$0.15 /GB outbound<br />10GB Database<br />$99.99 / month<br />Bandwidth<br />$0.10 /GB inbound<br />$0.15 /GB outbound<br />Specified by MAXSIZE on CREATE DATABASE command or portal (post-CTP1)<br />Monthly billing period<br />
    58. 58. SQL Azure and Windows Azure Table Comparison<br />Windows Azure Tables<br />SQL Azure Tables<br />Semi-structured<br />Loosely typed<br />Non-Relational (Not RDMS)<br />Massively scalable<br />Fully structured<br />Strongly typed<br />Relational (RDMS)<br />Highly scalable<br />
    59. 59. Added new Dynamic Management Views (DMVs) <br />Allows return of state information for health monitoring & performance tuning<br />sys.dm_exec_connections , _requests , _sessions, _tran_database_transactions, _active_transactions, _partition_stats<br />Added ability to move between Web or Business edition<br />ALTER DATABASE database_name { MODIFY (MAXSIZE = {1 | 10} GB) }<br />Increased idle session conn timeout (from 5 to 30 minutes)<br />Improves experience while using connection pooling and other interactive tools<br />Improved long running transaction algorithm <br />Allows import and export of much larger amounts of data without having to chunk your data <br />Recently Added (Feb 2010)<br />
    60. 60. SQL AzureOpportunities and Futures<br />Partitioned databases<br />Geo-location and geo-redundancy<br />Distributed query<br />Security w/AD, WLID, etc<br />Support for multiple levels of hardware and software isolation<br />BI features – SSRS, etc…<br />
    61. 61. Windows Azure Platform - http://www.azure.com/<br />Windows Azure Platform Training Kithttp://www.microsoft.com/downloads/details.aspx?FamilyID=413E88F8-5966-4A83-B309-53B7B77EDF78&displaylang=en<br />MSDN Development Centerhttp://msdn.microsoft.com/en-us/sqlserver/dataservices<br />Team Blog - http://blogs.msdn.com/ssds<br />Want to Know More?<br />
    62. 62. Lynn Langit<br />Blog: http://blogs.msdn.com/SoCalDevGal<br />Twitter: @llangit<br />Contact Me<br />