24 Hours of SQL Pass - SQL Azure


Published on

deck for 'Twitter Data on SQL Azure' presented March 2011

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Lynn
  • LynnMany solutions will use a hybrid (i.e. mixed) approach
  • LynnCode Far works with any type of code – i.e. Java, etc.. Does NOT have to be Azure or .NET
  • **note most search result streams are limited to 7 days**Search API LimitClients may request up to 1,500 statuses via the page and rpp parameters for the search method. The response to a request exceeding this limit will be a status code of 403 and the error message page parameter out of range in the format requested. This artificial limit is in place to ensure the performance of the search system. We also restrict the size of the search index by placing a date limit on the updates we allow you to search. This limit varies depending on the number of Tweets being created and the relevance of the results.Refreshing Search ResultsFor those using client side search widgets, by default the first request might include popular results. If you want to display these you can use the result_type attribute to visually differentiate them. If you don't want to display these you can always just pass the "result_type" parameter with a value of "recent" along with your request and they'll never be included.In the case that you are displaying them, you may not want to not redisplay them every time your widget refreshes. Each result payload will include the since_id that you should use in subsequent requests to get the latest results. If you are using this since_id (which all compliant search widgets should) then the popular results will be excluded in subsequent requests. The specific since_id value to pass up to the query string is included in the payload. Here are examples in various formats:
  • Sample code to handle re-try attempts for dropped connections - http://blogs.msdn.com/b/sqlazure/archive/2010/05/11/10011247.aspxSteve Hale (Microsoft) authored this article at http://blogs.msdn.com/sqlnativeclient/archive/2010/02/12/using-sql-server-client-apis-with-sql-azure-vversion-1-0.aspxAlso updated MSDN Dev Guide 'How to connect' http://msdn.microsoft.com/en-us/library/ee621787.aspxUsing SQL Server Client APIs with SQL Azure Version 1.0 This post describes some of the issues that the application developer should be aware of when writing ODBC, ADO.NET SQLClient, BCP, and PHP code that connects to SQL Azure. See also SQL Azure documentation on MSDN (http://msdn.microsoft.com/en-us/library/ee336279.aspx). Supported components1. Microsoft SQL Server Native Client 10.0 ODBC Driver or newer. This is a free download.2. Microsoft PHP 1.1 driver or newer. Again a free download.3. SQL Server 2008 BCP.EXE or newer. The caution here is that the queryout option is not fully supported for SQL Server 2008 RTM. Queryout is expected to be fully supported in a future release of SQL Server.4. ADO.NET 3.5 SqlClient or newer.Any other components are NOT currently supported for connecting to SQL Azure. In particular it should be noted that OLEDB connections are not currently supported from any OLEDB provider, even SQL Server 2008 Native Client OLEDB provider. ODBC connections using the on-the-box ODBC driver “SQL Server” are not currently supported.Connecting1. valid server name, user ID, and password for SQL Azure2. On the SQL Azure portal, set the firewall rules to allow your client IP address to connect to SQL Azure. Additionally, in some corporate environments, you might need to install Microsoft Firewall Client for ISA Server on the client machines or connections will not succeed. And in those environments, using IIS might require the default IIS settings for Application Pool Identity to be changed. Here is how that setting is changed: Open IIS Manager in Admin Tools, select Application Pools, Pick the pool, Advanced Setting and under Process Model change Identity from BuiltInAccount\\ApplicationPoolIdentity to Custom account domain account. Restart IIS.SQL Azure supports only TCP/IP connections using port 1433 (default for most SQL Server APIs) if other protocol choices (named pipes or using other ports) Additionally, most client APIs do not send the server name when connecting, thus the server name should be included as part of the user ID. Currently the only exception to this is .NET 4.0 which does send the server name. ODBC, BCP, PHP, and .NET 3.5 all require the server name to be sent as part of the user ID. This may change in a future release. Connection strings are all similar for each API stack and require the server name, user ID, password, and optionally the database. Trusted connections (also known as Integrated Security or Windows Authentication) are not supported. MARS is not yet supported by SQL Azure so make sure MARS is not enabled in the connection string.Creating and Connecting to a DatabaseSQLCmd example: sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword>Note the required server name (in yellow) included as part of the user ID.In the above example the database was not specified. In that case the default database “master” is used. You must connect to the master database to create databases or drop databases. Since the “USE database” command is not supported, you must disconnect after creating a database and reconnect directly to the user database. In this example we connect directly to the Northwind database.sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword> -d NorthwindCreating tablesTable creation is restricted to user databases; you cannot create tables in the master database. Table creation is substantially the same as on SQL Server, with the exception that certain data types are not supported, for example CLR User Defined Types, Filestream, and row guid types. Also, it should be mentioned that SQL Azure requires a clustered index or primary key (which uses a clustered index by default) before data can be inserted into a table. For this reason a clustered index or primary key must always be created for any table intended to contain data.Example:Create table employees(EmployeeIDint primary key, EmployeeNamevarchar(30))OrCreate table employees(EmployeeIDint, EmployeeNamevarchar(30))Create clustered index empIndex on employees(EmployeeID) Unsupported features· “USE database” is not supported· 3-part names are not supported unless the database being referenced is the current database.· Creating or dropping a database when not in the master database· Insertion of data into a table without a clustered index.· MARS connections· Trusted connections· CLR UDTs· XML schema collections· Distributed transactions (local transactions are supported)· AttachDB· Certain stored procedures, for example sp_who and undocumented stored procedures used to support OLEDB schema rowsets. Deprecated functionalitySome features are deprecated in SQL Server and should not be used in new application development. In most cases SQL Azure does not support deprecated SQL Server features. Some examples of these are:· System tables such as sysobjects, sysusers, or sysdatabases. Use the new system views instead. See http://msdn.microsoft.com/en-us/library/ms187997.aspx for a mapping to the new system views.· Numbered stored procedures· “select into” syntax· “COMPUTE BY” clause· Two-part drop index statements See http://msdn.microsoft.com/en-us/library/ee336281.aspx for more detailed Transact-SQL support.Connection behaviorSQL Azure connections will behave somewhat differently than SQL Server connections. They can be dropped for a number of reasons:· Excessive resource usage· Long-running queries· Long-running single transactions· Idle connections· Failover due to server failuresFor this reason it’s best to avoid caching connections for later use. A better strategy is to open the connection when needed, use it, and then close it. Additionally, it is recommended that applications provide retry/reconnect logic to recover from dropped connections.LatencyLatency may be a concern when using SQL Azure unless the SQL Azure server is located near the application (“code near” scenario), such as with Windows Azure and a co-located SQL Azure. Making many round trips to the server can introduce noticeable delay in an application that previously performed adequately when running against a SQL Server on the intranet. Ensure the application connects to the nearest SQL Azure server and that round trips are minimized. SUMMARYü Use only supported client components to connectü Connect using TCP/IP and port 1433ü Use SQL Authentication only, not Trusted Connections/Integrated Authentication/Windows Authentication.ü Specify the server name in the user ID.ü Ensure the firewall rules in the SQL Azure portal permit your IP address.ü Install Microsoft Firewall Client for ISA Server if needed.ü Change the IIS Application Pool Identity if needed.ü Make sure MARS is not enabled.ü Connect to the master database to create or drop databasesü Create a user database for your application since tables cannot be created in masterü Disconnect and reconnect to change databasesü Create a clustered index or primary key on a table before inserting data.ü Remove any unsupported or deprecated features from the application.ü Avoid caching connections. Open the connection, use it, and then close it.ü Avoid making numerous round trips to the server.
  • LynnClustered indexes sort and store the data rows in the table based on their key values. There can only be one clustered index per table, because the data rows themselves can only be sorted in one order. For more information about clustered index architectureWith few exceptions, every table should have a clustered index defined on the column, or columns, that offer the following: Can be used for frequently used queries. Provide a high degree of uniqueness. Can be used in range queries.Distributed Query – reference multiple linked servers and can perform either updae or read operations against each individual linked serverDistributed transactions span two or more servers known as resource managers. The management of the transaction must be coordinated between the resource managers by a server component called a transaction manager.DDL – data definition language statement to create drop or alter mining models and multidimensional objects such as cubes and dimensions
  • http://sqlazuremw.codeplex.com
  • Copy Database commandCREATE DATABASE DB2A AS COPY OF Server1.DB1A For MonitoringGRANT VIEW DATABASE STATE TO database_user;Example: Space UsedMonitor space used via ‘sys.dm_db_partition_stats’Trap ‘out of space exception’ – SQL Error 40544
  • https://www.sqlazurelabs.com/
  • http://www.silverlight.net/content/samples/odataexplorer/ - Odata viewer
  • Lynn talks
  • Lynn talks
  • Lynn talks
  • https://datamarket.azure.com/
  • Lynn talks
  • LynnMSDN article: http://msdn.microsoft.com/en-us/library/ff951624.aspxBlog about Federation: http://blogs.msdn.com/b/cbiyikoglu/archive/2010/10/30/building-scalable-database-solution-in-sql-azure-introducing-federation-in-sql-azure.aspxSupport for database copy: Database copy allows you to make a real-time complete snapshot of your database into a different server in the data center. This new copy feature is the first step in backup support for SQL Azure, allowing you to get a complete backup of any SQL Azure database before making schema or database changes to the source database. The ability to snapshot a database easily is our top requested feature for SQL Azure, and goes above and beyond our database center replication to keep your data always available. The MSDN Documentation with more information is entitled: Copying Databases in SQL Azure.Update on Project Houstonhttp://blogs.msdn.com/b/sqlazure/archive/2010/08/25/10054230.aspx
  • Lynn
  • Both
  • Lynn
  • Lynn
  • 24 Hours of SQL Pass - SQL Azure

    1. 1. Lynn Langithttp://blogs.msdn.com/SoCalDevGaltwitter - @llangit<br />
    2. 2. Windows Azure Platform<br />Compute: Virtualized compute environment <br />Storage: Durable, scalable, & available storage<br />Management: Automated, model-driven management<br />Database:Relational processing and services for structured/unstructured data – Data Marketplace<br />Service Bus: General purpose application bus<br />Access Control: Rules-driven, claims-based access control<br />
    3. 3. Windows Azure Platform<br />
    4. 4. Data Storage Choices<br />On-premise<br /> SQL Server or other s/w on-premise<br /> Resource governance @ machine<br /> Full h/w control – 100% compatibility<br />Roll your own HA/DR/scale<br />Dedicated<br />SQL Azure (RDBMS)<br />Hosted<br />Virtual DB server<br />Resource governance @ LDB<br /> Security @ LDB<br />Auto HA, Fault-Tolerance<br />Self-provisioning<br />High RDMS compatibility<br /> Hosted SQL Server or other<br />Roll-your-own HA/DR/scale<br /> Security @ DB Server/OS<br />Resources<br />Windows Azure <br />Storage<br />Tables<br />Queues<br /> Blobs (also VMs)<br />Shared<br />Low<br />Control<br />High<br />
    5. 5. Application Topologies<br />SQL Server<br />App code/ Tools<br />App code/ Tools<br />Application/ Browser<br /> Data Sync<br />Windows<br />Azure<br />Windows<br />Azure<br />App Code<br />(ASP.NET)<br />App Code<br />(ASP.NET)<br />T-SQL / TDS<br />App Code<br />(ASP.NET)<br />App Code<br />(ASP.NET)<br />TSQLTDS<br />TSQLTDS<br />SQL Azure <br />SQL Azure <br />SQL Azure <br />Code Far<br />Code Near<br />Hybrid<br />
    6. 6. Setup Accounts<br />Twitter account / hashtag (#24HOP)<br />SQL Azure Server with one Database<br />SQL Azure Reporting CTP <br />Use Tools<br />Azure portal <br />SQL Server 2008 R2 Tools<br />SSMS on premise with SSRS / Report Builder 3.0<br />Business Intelligence Development Studio<br />SQL Azure Migration Wizard (CodePlex)<br />Visual Studio 2010 Ultimate<br />Excel 2010 with Power Pivot<br />Development Process<br />
    7. 7. Using SQL Azure Services<br />Power Pivot<br />Power Pivot<br />Report Viewer Control<br />Report Viewer Control<br />SQL Azure Reporting<br />SQL Server Reporting<br />SQL Azure<br />SQL Server<br />Twitter Data<br />Twitter Data<br />
    8. 8. Getting the Twitter Data<br /><ul><li>General API information - http://dev.twitter.com/
    9. 9. Search API examples - http://dev.twitter.com/doc/get/search
    10. 10. Containing a hashtag - http://search.twitter.com/search.atom?q=%2324HOP
    11. 11. From a user – http://search.twitter.com/search.atom?q=from%3Llangit
    12. 12. Mentioning a User - http://search.twitter.com/search.atom?q=%40Llangit
    13. 13. Refresh (get latest results – JSON syntax)
    14. 14. "refresh_url":"?since_id=9290798834&q=i"
    15. 15. Consider adding a ‘login to Twitter button’
    16. 16. Required for public Twitter clients </li></li></ul><li>SQL Azure Server Portal<br />
    17. 17. SQL Portal for Database<br />
    18. 18. We used ADO.NET and ODBC (OLE DB n/a at this time)<br />Client libraries pre-installed in Azure roles<br />Support for ASP.NET controls<br />Best practices for connecting<br />Connect directly to ONE database (no USE)<br />May need to include <login>@<server><br />Encrypt login information<br />Use Retry logic (can use connection library) <br />Use connection pooling for efficiency<br />Connection Model<br />
    19. 19. RDMS Features Used<br /><ul><li>Tables, indexes and views
    20. 20. Stored Procedures
    21. 21. Triggers
    22. 22. Constraints
    23. 23. Table variables
    24. 24. Session temp tables
    25. 25. Spatial datatype</li></li></ul><li>SSMS 2008 R2<br />
    26. 26. SQL Migration Wizard<br />
    27. 27. COPY Database Command<br />
    28. 28. DacPac<br />
    29. 29. SQL Azure Data Sync Service <br /><ul><li>Scale-out read or read/write
    30. 30. Geo replication of data
    31. 31. Edge network data distribution
    32. 32. Content delivery networks
    33. 33. sqlazurelabs.com</li></ul>SQL Azure<br />Sync Group<br />SQL Server<br />SQL Azure<br />On-Premises Applications<br />
    34. 34. Data Sync - Demo<br />
    35. 35. Table Order in Sync<br />
    36. 36. Tracking Objects Created<br /><ul><li>Tracking Tables
    37. 37. On Source / on Destination
    38. 38. Triggers
    39. 39. Stored Procedures</li></li></ul><li>OData Support (Labs)<br />Open Data Protocol<br />ATOM PUB <br />JSON <br />HTTP<br />https://www.sqlazurelabs.com<br />
    40. 40. Odata Query & Results<br />
    41. 41. Microsoft applications<br />Connectivity into BI applications<br />SSRS, SSIS, SSAS<br />Connectivity into Office 2010 applications<br />Excel, Excel PowerPivot, SharePoint, etc…<br />Custom applications<br />Can use Windows Azure – not required<br />Can connect via any client with valid connection string<br />Application Integration with SQL Azure<br />
    42. 42. BIDS SQL Server 2008 R2<br />Report Builder 3.0<br />Report Development<br />
    43. 43. SQL Server Report Builder 3.0<br />
    44. 44. SQL Server BIDS 2008 R2<br />
    45. 45. SQL Azure Reporting CTP<br />
    46. 46. Office 2010<br />Excel <br />Excel Power Pivot<br />SharePoint<br />SQL Server 2008<br />SQL Server Reporting Services<br />Report Builder 3.0<br />SQL Server Integration Services<br />SQL Server Analysis Services<br />Visual Studio 2010<br /> Server Explorer<br />Entity Framework<br />DACPAC<br />Windows Azure Marketplace DataMarket<br />Microsoft Application Integration<br />
    47. 47. Tools – Visual Studio 2010<br />Report Viewer Control – ‘out of box’ or customized<br />
    48. 48. Two UX versions – WinForms or WebForms<br />Two processing modes – Local or Remote (Azure)<br />Pass credentials<br />Customize UX<br />Report Viewer Control<br />
    49. 49. Report Viewer Control Customization<br />
    50. 50. Current Betas<br />SQL Azure Reporting Services<br />Odata<br />SQL Azure Data Sync v 1<br />Other Future Betas<br />SQL Azure Data Sync v 2 <br />SQL Azure Integrations Services <br />SQL Azure Analysis Services<br />SQL Azure Federation<br />Future - SQL Azure CTPs<br />VOTE for the FUTURE<br />http://www.mygreatwindowsazureidea.com/pages/34685-sql-azure-feature-voting<br />
    51. 51. Code - CodePlex<br />Database script for SQL Azure<br />Visual Studio 2010 .sln for client with Report viewer<br />BIDS .sln for Report (.rdl)<br />Solution<br />My Blog<br />YouTube Video<br />MSDN Article<br />Repositories<br />Requires SQL Azure Reporting CTP <br />
    52. 52. Windows Azure Platform –here<br />Windows Azure Platform Training Kit – here <br />MSDN Development Center – here <br />Team Blogs – SQLAzure<br />Want to Know More?<br />
    53. 53. Lynn Langit<br />Blog: http://blogs.msdn.com/SoCalDevGal<br />Twitter: @llangit<br />Contact Information<br />
    54. 54. Teaching High School Girls T-SQL<br /><ul><li>Dating Game Database
    55. 55. T-SQL queries – hour 1 (WHERE), hour 2 (JOIN)
    56. 56. Artifacts on CodePlex, can use SQL Azure to host
    57. 57. http://datinggame.codeplex.com</li></li></ul><li>TeachingKidsProgramming.org<br /><ul><li>Do a Recipe  Teach a Kid
    58. 58. Free Courseware (recipes)
    59. 59. Ages 10 ++
    60. 60. Microsoft SmallBasic</li></li></ul><li>Professional Association for SQL Server<br />Thank you to our sponsor<br />
    61. 61. Save 25%: Register by April 12th<br />www.sqlpass.org/sqlrally<br />May 11-13, Orlando, FL<br />Register by March31st: save 40% and have the chance to win a cruise to Alaska! <br />www.sqlpass.org/summit<br />Oct 11-14, Seattle, WA<br />