24 Hours of SQL Pass - SQL Azure

  • 2,063 views
Uploaded on

deck for 'Twitter Data on SQL Azure' presented March 2011

deck for 'Twitter Data on SQL Azure' presented March 2011

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,063
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
40
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Lynn
  • LynnMany solutions will use a hybrid (i.e. mixed) approach
  • LynnCode Far works with any type of code – i.e. Java, etc.. Does NOT have to be Azure or .NET
  • **note most search result streams are limited to 7 days**Search API LimitClients may request up to 1,500 statuses via the page and rpp parameters for the search method. The response to a request exceeding this limit will be a status code of 403 and the error message page parameter out of range in the format requested. This artificial limit is in place to ensure the performance of the search system. We also restrict the size of the search index by placing a date limit on the updates we allow you to search. This limit varies depending on the number of Tweets being created and the relevance of the results.Refreshing Search ResultsFor those using client side search widgets, by default the first request might include popular results. If you want to display these you can use the result_type attribute to visually differentiate them. If you don't want to display these you can always just pass the "result_type" parameter with a value of "recent" along with your request and they'll never be included.In the case that you are displaying them, you may not want to not redisplay them every time your widget refreshes. Each result payload will include the since_id that you should use in subsequent requests to get the latest results. If you are using this since_id (which all compliant search widgets should) then the popular results will be excluded in subsequent requests. The specific since_id value to pass up to the query string is included in the payload. Here are examples in various formats:
  • Sample code to handle re-try attempts for dropped connections - http://blogs.msdn.com/b/sqlazure/archive/2010/05/11/10011247.aspxSteve Hale (Microsoft) authored this article at http://blogs.msdn.com/sqlnativeclient/archive/2010/02/12/using-sql-server-client-apis-with-sql-azure-vversion-1-0.aspxAlso updated MSDN Dev Guide 'How to connect' http://msdn.microsoft.com/en-us/library/ee621787.aspxUsing SQL Server Client APIs with SQL Azure Version 1.0 This post describes some of the issues that the application developer should be aware of when writing ODBC, ADO.NET SQLClient, BCP, and PHP code that connects to SQL Azure. See also SQL Azure documentation on MSDN (http://msdn.microsoft.com/en-us/library/ee336279.aspx). Supported components1. Microsoft SQL Server Native Client 10.0 ODBC Driver or newer. This is a free download.2. Microsoft PHP 1.1 driver or newer. Again a free download.3. SQL Server 2008 BCP.EXE or newer. The caution here is that the queryout option is not fully supported for SQL Server 2008 RTM. Queryout is expected to be fully supported in a future release of SQL Server.4. ADO.NET 3.5 SqlClient or newer.Any other components are NOT currently supported for connecting to SQL Azure. In particular it should be noted that OLEDB connections are not currently supported from any OLEDB provider, even SQL Server 2008 Native Client OLEDB provider. ODBC connections using the on-the-box ODBC driver “SQL Server” are not currently supported.Connecting1. valid server name, user ID, and password for SQL Azure2. On the SQL Azure portal, set the firewall rules to allow your client IP address to connect to SQL Azure. Additionally, in some corporate environments, you might need to install Microsoft Firewall Client for ISA Server on the client machines or connections will not succeed. And in those environments, using IIS might require the default IIS settings for Application Pool Identity to be changed. Here is how that setting is changed: Open IIS Manager in Admin Tools, select Application Pools, Pick the pool, Advanced Setting and under Process Model change Identity from BuiltInAccount\\ApplicationPoolIdentity to Custom account domain account. Restart IIS.SQL Azure supports only TCP/IP connections using port 1433 (default for most SQL Server APIs) if other protocol choices (named pipes or using other ports) Additionally, most client APIs do not send the server name when connecting, thus the server name should be included as part of the user ID. Currently the only exception to this is .NET 4.0 which does send the server name. ODBC, BCP, PHP, and .NET 3.5 all require the server name to be sent as part of the user ID. This may change in a future release. Connection strings are all similar for each API stack and require the server name, user ID, password, and optionally the database. Trusted connections (also known as Integrated Security or Windows Authentication) are not supported. MARS is not yet supported by SQL Azure so make sure MARS is not enabled in the connection string.Creating and Connecting to a DatabaseSQLCmd example: sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword>Note the required server name (in yellow) included as part of the user ID.In the above example the database was not specified. In that case the default database “master” is used. You must connect to the master database to create databases or drop databases. Since the “USE database” command is not supported, you must disconnect after creating a database and reconnect directly to the user database. In this example we connect directly to the Northwind database.sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword> -d NorthwindCreating tablesTable creation is restricted to user databases; you cannot create tables in the master database. Table creation is substantially the same as on SQL Server, with the exception that certain data types are not supported, for example CLR User Defined Types, Filestream, and row guid types. Also, it should be mentioned that SQL Azure requires a clustered index or primary key (which uses a clustered index by default) before data can be inserted into a table. For this reason a clustered index or primary key must always be created for any table intended to contain data.Example:Create table employees(EmployeeIDint primary key, EmployeeNamevarchar(30))OrCreate table employees(EmployeeIDint, EmployeeNamevarchar(30))Create clustered index empIndex on employees(EmployeeID) Unsupported features· “USE database” is not supported· 3-part names are not supported unless the database being referenced is the current database.· Creating or dropping a database when not in the master database· Insertion of data into a table without a clustered index.· MARS connections· Trusted connections· CLR UDTs· XML schema collections· Distributed transactions (local transactions are supported)· AttachDB· Certain stored procedures, for example sp_who and undocumented stored procedures used to support OLEDB schema rowsets. Deprecated functionalitySome features are deprecated in SQL Server and should not be used in new application development. In most cases SQL Azure does not support deprecated SQL Server features. Some examples of these are:· System tables such as sysobjects, sysusers, or sysdatabases. Use the new system views instead. See http://msdn.microsoft.com/en-us/library/ms187997.aspx for a mapping to the new system views.· Numbered stored procedures· “select into” syntax· “COMPUTE BY” clause· Two-part drop index statements See http://msdn.microsoft.com/en-us/library/ee336281.aspx for more detailed Transact-SQL support.Connection behaviorSQL Azure connections will behave somewhat differently than SQL Server connections. They can be dropped for a number of reasons:· Excessive resource usage· Long-running queries· Long-running single transactions· Idle connections· Failover due to server failuresFor this reason it’s best to avoid caching connections for later use. A better strategy is to open the connection when needed, use it, and then close it. Additionally, it is recommended that applications provide retry/reconnect logic to recover from dropped connections.LatencyLatency may be a concern when using SQL Azure unless the SQL Azure server is located near the application (“code near” scenario), such as with Windows Azure and a co-located SQL Azure. Making many round trips to the server can introduce noticeable delay in an application that previously performed adequately when running against a SQL Server on the intranet. Ensure the application connects to the nearest SQL Azure server and that round trips are minimized. SUMMARYü Use only supported client components to connectü Connect using TCP/IP and port 1433ü Use SQL Authentication only, not Trusted Connections/Integrated Authentication/Windows Authentication.ü Specify the server name in the user ID.ü Ensure the firewall rules in the SQL Azure portal permit your IP address.ü Install Microsoft Firewall Client for ISA Server if needed.ü Change the IIS Application Pool Identity if needed.ü Make sure MARS is not enabled.ü Connect to the master database to create or drop databasesü Create a user database for your application since tables cannot be created in masterü Disconnect and reconnect to change databasesü Create a clustered index or primary key on a table before inserting data.ü Remove any unsupported or deprecated features from the application.ü Avoid caching connections. Open the connection, use it, and then close it.ü Avoid making numerous round trips to the server.
  • LynnClustered indexes sort and store the data rows in the table based on their key values. There can only be one clustered index per table, because the data rows themselves can only be sorted in one order. For more information about clustered index architectureWith few exceptions, every table should have a clustered index defined on the column, or columns, that offer the following: Can be used for frequently used queries. Provide a high degree of uniqueness. Can be used in range queries.Distributed Query – reference multiple linked servers and can perform either updae or read operations against each individual linked serverDistributed transactions span two or more servers known as resource managers. The management of the transaction must be coordinated between the resource managers by a server component called a transaction manager.DDL – data definition language statement to create drop or alter mining models and multidimensional objects such as cubes and dimensions
  • http://sqlazuremw.codeplex.com
  • Copy Database commandCREATE DATABASE DB2A AS COPY OF Server1.DB1A For MonitoringGRANT VIEW DATABASE STATE TO database_user;Example: Space UsedMonitor space used via ‘sys.dm_db_partition_stats’Trap ‘out of space exception’ – SQL Error 40544
  • https://www.sqlazurelabs.com/
  • http://www.silverlight.net/content/samples/odataexplorer/ - Odata viewer
  • Lynn talks
  • Lynn talks
  • Lynn talks
  • https://datamarket.azure.com/
  • Lynn talks
  • LynnMSDN article: http://msdn.microsoft.com/en-us/library/ff951624.aspxBlog about Federation: http://blogs.msdn.com/b/cbiyikoglu/archive/2010/10/30/building-scalable-database-solution-in-sql-azure-introducing-federation-in-sql-azure.aspxSupport for database copy: Database copy allows you to make a real-time complete snapshot of your database into a different server in the data center. This new copy feature is the first step in backup support for SQL Azure, allowing you to get a complete backup of any SQL Azure database before making schema or database changes to the source database. The ability to snapshot a database easily is our top requested feature for SQL Azure, and goes above and beyond our database center replication to keep your data always available. The MSDN Documentation with more information is entitled: Copying Databases in SQL Azure.Update on Project Houstonhttp://blogs.msdn.com/b/sqlazure/archive/2010/08/25/10054230.aspx
  • Lynn
  • Both
  • Lynn
  • Lynn

Transcript

  • 1. Lynn Langithttp://blogs.msdn.com/SoCalDevGaltwitter - @llangit
  • 2. Windows Azure Platform
    Compute: Virtualized compute environment
    Storage: Durable, scalable, & available storage
    Management: Automated, model-driven management
    Database:Relational processing and services for structured/unstructured data – Data Marketplace
    Service Bus: General purpose application bus
    Access Control: Rules-driven, claims-based access control
  • 3. Windows Azure Platform
  • 4. Data Storage Choices
    On-premise
    SQL Server or other s/w on-premise
    Resource governance @ machine
    Full h/w control – 100% compatibility
    Roll your own HA/DR/scale
    Dedicated
    SQL Azure (RDBMS)
    Hosted
    Virtual DB server
    Resource governance @ LDB
    Security @ LDB
    Auto HA, Fault-Tolerance
    Self-provisioning
    High RDMS compatibility
    Hosted SQL Server or other
    Roll-your-own HA/DR/scale
    Security @ DB Server/OS
    Resources
    Windows Azure
    Storage
    Tables
    Queues
    Blobs (also VMs)
    Shared
    Low
    Control
    High
  • 5. Application Topologies
    SQL Server
    App code/ Tools
    App code/ Tools
    Application/ Browser
    Data Sync
    Windows
    Azure
    Windows
    Azure
    App Code
    (ASP.NET)
    App Code
    (ASP.NET)
    T-SQL / TDS
    App Code
    (ASP.NET)
    App Code
    (ASP.NET)
    TSQLTDS
    TSQLTDS
    SQL Azure
    SQL Azure
    SQL Azure
    Code Far
    Code Near
    Hybrid
  • 6. Setup Accounts
    Twitter account / hashtag (#24HOP)
    SQL Azure Server with one Database
    SQL Azure Reporting CTP
    Use Tools
    Azure portal
    SQL Server 2008 R2 Tools
    SSMS on premise with SSRS / Report Builder 3.0
    Business Intelligence Development Studio
    SQL Azure Migration Wizard (CodePlex)
    Visual Studio 2010 Ultimate
    Excel 2010 with Power Pivot
    Development Process
  • 7. Using SQL Azure Services
    Power Pivot
    Power Pivot
    Report Viewer Control
    Report Viewer Control
    SQL Azure Reporting
    SQL Server Reporting
    SQL Azure
    SQL Server
    Twitter Data
    Twitter Data
  • 8. Getting the Twitter Data
    • General API information - http://dev.twitter.com/
    • 9. Search API examples - http://dev.twitter.com/doc/get/search
    • 10. Containing a hashtag - http://search.twitter.com/search.atom?q=%2324HOP
    • 11. From a user – http://search.twitter.com/search.atom?q=from%3Llangit
    • 12. Mentioning a User - http://search.twitter.com/search.atom?q=%40Llangit
    • 13. Refresh (get latest results – JSON syntax)
    • 14. "refresh_url":"?since_id=9290798834&q=i"
    • 15. Consider adding a ‘login to Twitter button’
    • 16. Required for public Twitter clients
  • SQL Azure Server Portal
  • 17. SQL Portal for Database
  • 18. We used ADO.NET and ODBC (OLE DB n/a at this time)
    Client libraries pre-installed in Azure roles
    Support for ASP.NET controls
    Best practices for connecting
    Connect directly to ONE database (no USE)
    May need to include <login>@<server>
    Encrypt login information
    Use Retry logic (can use connection library)
    Use connection pooling for efficiency
    Connection Model
  • 19. RDMS Features Used
  • SSMS 2008 R2
  • 26. SQL Migration Wizard
  • 27. COPY Database Command
  • 28. DacPac
  • 29. SQL Azure Data Sync Service
    • Scale-out read or read/write
    • 30. Geo replication of data
    • 31. Edge network data distribution
    • 32. Content delivery networks
    • 33. sqlazurelabs.com
    SQL Azure
    Sync Group
    SQL Server
    SQL Azure
    On-Premises Applications
  • 34. Data Sync - Demo
  • 35. Table Order in Sync
  • 36. Tracking Objects Created
    • Tracking Tables
    • 37. On Source / on Destination
    • 38. Triggers
    • 39. Stored Procedures
  • OData Support (Labs)
    Open Data Protocol
    ATOM PUB
    JSON
    HTTP
    https://www.sqlazurelabs.com
  • 40. Odata Query & Results
  • 41. Microsoft applications
    Connectivity into BI applications
    SSRS, SSIS, SSAS
    Connectivity into Office 2010 applications
    Excel, Excel PowerPivot, SharePoint, etc…
    Custom applications
    Can use Windows Azure – not required
    Can connect via any client with valid connection string
    Application Integration with SQL Azure
  • 42. BIDS SQL Server 2008 R2
    Report Builder 3.0
    Report Development
  • 43. SQL Server Report Builder 3.0
  • 44. SQL Server BIDS 2008 R2
  • 45. SQL Azure Reporting CTP
  • 46. Office 2010
    Excel
    Excel Power Pivot
    SharePoint
    SQL Server 2008
    SQL Server Reporting Services
    Report Builder 3.0
    SQL Server Integration Services
    SQL Server Analysis Services
    Visual Studio 2010
    Server Explorer
    Entity Framework
    DACPAC
    Windows Azure Marketplace DataMarket
    Microsoft Application Integration
  • 47. Tools – Visual Studio 2010
    Report Viewer Control – ‘out of box’ or customized
  • 48. Two UX versions – WinForms or WebForms
    Two processing modes – Local or Remote (Azure)
    Pass credentials
    Customize UX
    Report Viewer Control
  • 49. Report Viewer Control Customization
  • 50. Current Betas
    SQL Azure Reporting Services
    Odata
    SQL Azure Data Sync v 1
    Other Future Betas
    SQL Azure Data Sync v 2
    SQL Azure Integrations Services
    SQL Azure Analysis Services
    SQL Azure Federation
    Future - SQL Azure CTPs
    VOTE for the FUTURE
    http://www.mygreatwindowsazureidea.com/pages/34685-sql-azure-feature-voting
  • 51. Code - CodePlex
    Database script for SQL Azure
    Visual Studio 2010 .sln for client with Report viewer
    BIDS .sln for Report (.rdl)
    Solution
    My Blog
    YouTube Video
    MSDN Article
    Repositories
    Requires SQL Azure Reporting CTP
  • 52. Windows Azure Platform –here
    Windows Azure Platform Training Kit – here
    MSDN Development Center – here
    Team Blogs – SQLAzure
    Want to Know More?
  • 53. Lynn Langit
    Blog: http://blogs.msdn.com/SoCalDevGal
    Twitter: @llangit
    Contact Information
  • 54. Teaching High School Girls T-SQL
    • Dating Game Database
    • 55. T-SQL queries – hour 1 (WHERE), hour 2 (JOIN)
    • 56. Artifacts on CodePlex, can use SQL Azure to host
    • 57. http://datinggame.codeplex.com
  • TeachingKidsProgramming.org
    • Do a Recipe  Teach a Kid
    • 58. Free Courseware (recipes)
    • 59. Ages 10 ++
    • 60. Microsoft SmallBasic
  • Professional Association for SQL Server
    Thank you to our sponsor
  • 61. Save 25%: Register by April 12th
    www.sqlpass.org/sqlrally
    May 11-13, Orlando, FL
    Register by March31st: save 40% and have the chance to win a cruise to Alaska!
    www.sqlpass.org/summit
    Oct 11-14, Seattle, WA