Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SQL under the hood


Published on

In this presentation we review some internal process inside SQL Azure.


Ing. Eduardo Castro, PhD

  • Be the first to comment

SQL under the hood

  1. 1. SQL Azure DatabaseUnder the hood<br />Ing. Eduardo Castro, PhD<br />Comunidad Windows<br /><br /><br />
  2. 2. Agenda<br />Service Review<br />SQL Azure Architecture & Workflows<br />Service Resilience<br />Service Monitoring <br />Attack Vectors/Security considerations<br />Wrap up<br />
  3. 3. What is “SQL Azure”?<br />
  4. 4. The Azure Services PlaformAn illustration<br />.NET Services<br />SQL Azure<br />Applications<br />Windows Azure<br />Applications<br />Windows<br />Mobile<br />Windows<br />Vista/XP<br />Windows<br />Server<br />Others<br />
  5. 5. Review – Conceptual model<br />Subscription <br />Used to map service usage to the billing instrument<br />Users may have many subscriptions<br />Logical Server<br />Akin to SQL Server Instance<br />Unit of Geo-Location & Billing<br />1:1 Subscription & server<br />User Database<br />Restricted T-SQL surface area<br />Additional catalog views provided e.g. sys.billing, sys.firewall_rules, etc<br />
  6. 6. SQL AzureA relational DB in the cloud<br />SQL Azure Database<br />Data Hub<br />Others (Future)<br />Relational database as a service<br />Highly available, automatically maintained<br />Extension of the SQL Server Data Platform<br />.NET Services<br />SQL Services<br />Applications<br />Live Services<br />Windows Azure<br />Applications<br />Windows<br />Mobile<br />Windows<br />Vista/XP<br />Windows<br />Server<br />Others<br />
  7. 7. Extending SQL Server Data Platform to the Cloud<br />Data Sync<br />Reference Data<br />Database<br />Symmetric Programming Model<br />Data Hub Aggregation<br /><ul><li>Initial services – core RDBMS capabilities with SQL Azure Database, Data Sync
  8. 8. Future Offerings
  9. 9. Additional data platform capabilities: Reporting, BI
  10. 10. New services: Reference Data</li></li></ul><li>The New SQL Data Services<br />Clear Feedback: “I want a database in the Cloud”<br />Familiar SQL Server relational model<br />Uses existing APIs & tools<br />Built for the Cloud with availability and scale<br />Accessible to all from PHP, Ruby, and Java<br />Focus on combining the best features of SQL Server running at scale with low friction<br />
  11. 11. The Evolution of SDS<br />Evolves<br />BrowserApplication<br />Application<br />Application<br />BrowserApplication<br />Application<br />ODBC, OLEDB, ADO.Net PHP, Ruby, …<br />REST Client<br />SQL Client*<br />REST Client<br />Cloud<br />Cloud<br />Windows Azure<br />REST (Astoria)<br />Web App<br />ADO.Net + EF<br />REST Client<br />HTTP+REST<br />HTTP+REST<br />HTTP<br />TDS<br />HTTP<br />Windows Azure<br />Web App<br />SQL Client*<br />Data Center<br />Data Center<br />TDS + TSQL Model<br />REST/SOAP + ACE Model<br />SDS Next<br />SDS Current<br />* Client access enabled using TDS for ODBC, ADO.Net, OLEDB, PHP-SQL, Ruby, …<br />
  12. 12. SQL Azure Network Topology<br />Applications use standard SQL client libraries: ODBC, ADO.Net, PHP, …<br />Application<br />Internet<br />Azure Cloud<br />TDS (tcp)<br />Security Boundary<br />Load balancer forwards ‘sticky’ sessions to TDS protocol tier<br />LB<br />TDS (tcp)<br />Gateway<br />Gateway<br />Gateway<br />Gateway<br />Gateway<br />Gateway<br />Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to CloudDB<br />TDS (tcp)<br />SQL<br />SQL<br />SQL<br />SQL<br />SQL<br />SQL<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />
  13. 13. TDS Gateway<br />TDS Listener<br />Capability negotiation<br />TDS Packet inspection<br />Security<br />Logical->Physical mapping via metadata catalog<br />Enabler for multi-tenet capabilities<br />Isolation layer<br />
  14. 14. TDS Gateway Layering<br />Gateway Process<br />TDS Endpoint<br />AdminSvc Endpoint<br />Provisioning Endpoint<br />Protocol Parser<br />Business Logic Services<br />Connection Mgmt<br />SQL<br />SQL<br />SQL<br />SQL<br />SQL<br />SQL<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />
  15. 15. Provisioning <br />Subscription<br />Coordinated across all Azure services<br />Executed in parallel w/retries<br />Server<br />May occur between data centers<br />Point where Geo-location is established<br />Database<br />Always occurs within a single data center<br />Cross node operations executed during this process e.g. add new db to sys.databases on the master<br />
  16. 16. Server Provisioning<br />Driven by administrator Portal<br />Provision request is sent to Gateway<br />Metadata catalog entry created<br />DNS record (CNAME) created within LiveDNS service<br />Master DB created<br />On completion metadata catalog updated<br />
  17. 17. SQL Azure Server Provisioning<br />Live DNS Cluster<br />Customer Browser<br />Live DNS Svc<br />Datacenter (Sub-Region)<br />1<br />5<br />Portal LB<br />Gateway LB<br />2<br />4<br />3<br />6<br />Front-end Node<br />Front-end Node<br />Front-end Node<br />Front-end Node<br />Gateway <br />Gateway <br />Admin Portal <br />Admin Portal <br />7<br />Backend Node<br />Backend Node<br />Backend Node<br />SQL Server<br />SQL Server<br />SQL Server<br />Mgmt. Services<br />Mgmt. Services<br />Mgmt. Services<br />Fabric<br />Fabric<br />Fabric<br />
  18. 18. Database Provisioning<br />Gateway performs stateful TDS packet inspection<br />Picks out subset of messages<br />Parses out args for create database<br />Makes entry into Gateway metadata catalog<br />Unused replica set located and reserved<br />Replica set (UserDB) is prepped for use <br />Metadata catalog is updated<br />
  19. 19. SQL Azure Database provisioning<br />TDS Gateway<br />1<br />Front-end Node<br />Protocol Parser<br />TDS Session<br />2<br />3<br />Gateway Logic<br />Master Node<br />Master Cluster<br />Master Node Components<br />4<br />7<br />5<br />6<br />8<br />Backend Node 1<br />Backend Node 2<br />Backend Node 3<br />SQL Instance<br />SQL Instance<br />SQL Instance<br />SQL DB<br />SQL DB<br />SQL DB<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />
  20. 20. SQL Azure Login Process<br />Login request arrives at the Gateway<br />Gateway locates MasterDb & UserDb replica sets<br />Credentials are validated against MasterDb<br />TDS session is opened to UserDB and requests are forwarded<br />
  21. 21. SQL Azure Login Process<br />TDS Gateway<br />7<br />1<br />Front-end Node<br />Protocol Parser<br />TDS Session<br />2<br />6<br />Gateway Logic<br />Master Node<br />Global Partition Map<br />Master Node Components<br />3<br />8<br />4<br />5<br />Backend Node 1<br />Backend Node 2<br />Backend Node 3<br />SQL Instance<br />SQL Instance<br />SQL Instance<br />SQL DB<br />SQL DB<br />SQL DB<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />
  22. 22. Service Resilience<br />Provisioning<br />State machines used to coordinate activities across node (and datacenter) boundaries<br />Failed provisioning attempts cleaned automatically after 10 minutes<br />Login<br />Failovers during the login will be transparent (<30 seconds)<br />Metadata catalog refresh occurs automatically<br />Active Session<br />Surface as connection drops (due to state)<br />
  23. 23. Monitoring Service Health<br />Metrics<br />Cluster wide performance counters gather key metrics on the service<br />Used to alert Operations to issues before they become a problem<br />Early warning system<br />Code issues<br />Capacity warnings<br />Health<br />Exercises the service routinely looking for problems<br />When issues are encountered runs deep diagnostics<br />Network connectivity at the node level<br />Validate all dependent services (Live DNS, Live ID, etc)<br />Monitoring from other MSFT DC’s<br />Validates accessibility from multiple geographic locations<br />Alerts fired automatically when test jobs fail<br />
  24. 24. Security/Attack Considerations<br />Service <br />Secure channel required (SSL)<br />Denial Of Service trend tracking<br />Packet Inspection<br />Server <br />IP allow list (Firewall) <br />Idle connection culling<br />Generated server names <br />Database<br />Disallow the most commonly attacked user id’s (SA, Admin, root, guest, etc) <br />Standard SQL Authn/Authz mode <br />
  25. 25. Wrap Up<br />Reviewed SQL Azure Architecture & Workflows<br />Provisioning (Server & DB)<br />Login<br />Service Resilience & Health<br />Failure detection and correction<br />How we determine service health<br />Security considerations<br />Attack vectors and mitigations <br />Questions?<br />
  26. 26. Links<br /><br /><br /><br /><br /><br />
  27. 27. Q&A<br />