Roles and Responsibilities Managing a Microsoft SQL AzureIng. Eduardo Castro, PhDGrupoAsesor en Informáticaecastro@grupoasesor.net
AgendaIntroduction to SQL Data Services (SDS)SDS ConceptsDemo: Provisioning of SDS Server and DatabaseManaging SDS Database
The New SQL Data ServicesDatabase as a serviceFocus on combining the best features of SQL Server running at scale with ease of useHigh compatibility with existing SQL Server offeringIn V1, addresses web/departmental workloads
Data Platform: Device to CloudRDBMSServiceMiningProtectionSyncLoadReportingCachingQueryAnalysisIntegrationSearchBackupModel & policy based development and managementInMemoryRelationalBLOBMultiDimXMLFileTypePlaceMega-serverDesktopData ServicesServerMobile/Desktop
Database ChoicesValue Props:Full h/w control – size/scale100% of API surface areaRoll-your-own HA/DR/scaleValue Props:100% of API surface areaRoll-your-own HA/DR/scaleDedicatedOn-premise SQL Server or other s/w on-premise Resource governance @ machine Security @ DB Server/OSValue Props:Auto HA, Fault-ToleranceFriction-free scaleSelf-provisioningSubset of API surface areaResourcesHosted Hosted SQL Server Resource governance @ VM Security @ DB Server/OSSDS Virtual DB serverResource governance @ DB Security @ DB/Virtual ServerSharedLow“Friction”/ControlHighSDS V1 targets scenarios that live in the lower left quadrant
SDS ConceptsProvisioning modelAccount, server, databaseTraditional SQL security modelAuthenticate logins, map to users and rolesAuthorize users and roles to SQL objectsFamiliar SQL programming modelTraditional SQL Server relational modelSupport ADO.Net, ODBCApplications must be partition aware for larger databases
Azure™ Services PlatformAzure Services Platform
The Microsoft Platform StackMobileDesktopServerCloudApplicationDataOS
Extending SQL Data Platform to CloudReference DataBusiness IntelligenceData SyncReportingSQL Data Services (SDS)Database-as-a-ServiceSymmetric Programming ModelData Hub AggregationInitial services – core RDBMS capabilities as a service (SDS), Data Sync and Data Hub
Future Offerings
Additional data platform capabilities as a service: BI/DSS, DW
New services: Reference Data, Secure Data Hub
Enable new uses of data to deliver differentiated business valueService Provisioning ModelEach account has zero or more serversAzure wide, provisioned in a common portalBilling instrumentEach server has one or more databasesContains metadata about the databasesUnit of authenticationUnit of Geo-locationEach database has standard SQL objectsUnit of consistencyContains users, tables, views, indices, etc…     Account     Server       Database
SDS Server and Database Provisioningdemo
ArchitectureShared SQL infrastructureEach user database is replicated to multiple serversClient requests are routed to current “primary database”State-of-the-art HA technologyAutomatic failure detection; client request re-routed to new primaryLoad balancing across shared resource poolGateway provides TDS entry point, provisioning capabilityMachine 2Machine 3Machine 4Machine 1SQL InstanceSQL InstanceSQL InstanceSQL InstanceSQL DBSQL DBSQL DBSQL DBUserDB1UserDB3UserDB4UserDB2UserDB1UserDB2UserDB3UserDB4UserDB5UserDB2UserDB3UserDB4UserDB1UserDB5UserDB3UserDB4TDS entry point, provisioningHigh Availability Fabric: Failover and  Load balancing
High Level ArchitectureSQL Data Services Gateway TierProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesSQL Data Services Back EndSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. Services
Data Hub	“An aggregation of Enterprise, Partner, Desktop, and Device data within SQL Data Services”Windows AzureEnterpriseAzure AppOn-premises AppADO.NET, ODBC, PHP, ...SQL DataServices Data Sync Service SyncEnterpriseSyncSyncUser ManagementDeviceDevice ManagementDesktopClient AppClient AppSync GatewayBusiness Logic / RulesSync ClientSync Client
Data Hub – Project “Huron”Customer SegmentSmall and medium sized businessesEnterprise – rapid prototype Both Microsoft (v1) and Non-Microsoft device (post v1)Common PatternsMobile device access to corporate dataBusiness to business sharingSecure digital data hub / partner extranets Governmental data (searchable encryption)
Data Sync ServiceAnalyticalSync FrameworkTDSReportingSDS ProviderSQL Data ServicesData SyncOnlinePublish APISubscribe APISync API3rd party platformsClient Library APIClient Library APIWindowsDesktops& DevicesSync FrameworkSync FrameworkSync Service ProxySync Service Proxy3rd party DBClient Data ProviderClient Data Provider
SQL Data Services Network TopologyApplications use standard SQL client libraries: ODBC, OLEDB, ADO.Net, …ApplicationTDS (tcp:1433)Load balancer forwards ‘sticky’ sessions to TDS protocol tierLoad BalancerTDS (tcp: 1433)GatewayGatewayGatewayGatewayGatewayGatewayData NodeData NodeData NodeData NodeData NodeData NodeTDS (tcp: 1433)Scalability and Availability: Fabric, Failover, Replication and Load balancing
Scaling Out With SQL Data ServicesHow can I gain maximum performance from my data tier?What if my application has large data storage requirements?Scaling out with SQL Data Services addresses these requirements
An Example Scale Out ArchitectureASP.NetApplicationCustomer 123PartitionAwareApp TierDataPartitions“Shards”
Software Installation and PatchingSDS ResponsibilitiesSoftware installation, patching and upgradeMinimum interruption to serviceDBA ResponsibilitiesInstall, upgrade your own schemaSupport application and multi-server management model
Monitoring and Problem ResolutionSDS ResponsibilitiesService health is continuously monitoredHardware, availability, resource usage and etcSelf-healingAutomatic failoverLoad-balancingDedicated MS operation team 24/7Service dashboard for up-to-date health statusDBA ResponsibilitiesRelax and just use the service
High Availability and Disaster RecoverySDS ResponsibilitiesProvide automatic local HA capabilityPost V1, provide DR capabilityDatabase is automatically backed up with easy recovery for system errorsDBA ResponsibilitiesExport of data to protect against user errors“Clone” a databasePost V1, make available “self-service” restore
Database Admin Compatibility  GoalsSupport common application patterns and TSQL functionalitySupport common DBA tasksFocus on logical data administrationPhysical administration provided by service
Logical AdministrationDatabase Schema DesignDatabase User and Permission ManagementIndex ManagementQuery TuningStatistics ManagementData Import/ExportReporting…DBA knowledge is compatible with on-premise SQL
Logical vs. Physical AdministrationSupportedCreate/Drop databaseCreate/Drop tableCreate/Drop userIndex rebuildUpdate statistics…Not SupportedDatabase file placementFilegroup managementServer configuration optionsMirroring…
Example: CREATE TABLECREATE TABLE [[[ database_name . ] schema_name . table_name        ( { <column_definition> | 				     <computed_column_definition> |	     <column_set_definition> }        [ < table_constraint> ] [ ,...n ] ) [ ON { partition_scheme_name … | filegroup | "default" } ]   [ TEXTIMAGE_ON … ] [WITH ( < table_option > [ ,...n ] ) ]…No support for placement ex: FILEGROUP, TEXTIMAGE
Performance TroubleshootingDetect blocking query and deadlocksQuery optimizationTop N expensive queriesExcessive recompile, bad query planIndex tuningSome of existing SQL facilities will continue to workSET SHOWPLANSET STATISTICS DMV/DMF like system views…

Roles y Responsabilidades en SQL Azure

  • 1.
    Roles and ResponsibilitiesManaging a Microsoft SQL AzureIng. Eduardo Castro, PhDGrupoAsesor en Informáticaecastro@grupoasesor.net
  • 2.
    AgendaIntroduction to SQLData Services (SDS)SDS ConceptsDemo: Provisioning of SDS Server and DatabaseManaging SDS Database
  • 3.
    The New SQLData ServicesDatabase as a serviceFocus on combining the best features of SQL Server running at scale with ease of useHigh compatibility with existing SQL Server offeringIn V1, addresses web/departmental workloads
  • 4.
    Data Platform: Deviceto CloudRDBMSServiceMiningProtectionSyncLoadReportingCachingQueryAnalysisIntegrationSearchBackupModel & policy based development and managementInMemoryRelationalBLOBMultiDimXMLFileTypePlaceMega-serverDesktopData ServicesServerMobile/Desktop
  • 5.
    Database ChoicesValue Props:Fullh/w control – size/scale100% of API surface areaRoll-your-own HA/DR/scaleValue Props:100% of API surface areaRoll-your-own HA/DR/scaleDedicatedOn-premise SQL Server or other s/w on-premise Resource governance @ machine Security @ DB Server/OSValue Props:Auto HA, Fault-ToleranceFriction-free scaleSelf-provisioningSubset of API surface areaResourcesHosted Hosted SQL Server Resource governance @ VM Security @ DB Server/OSSDS Virtual DB serverResource governance @ DB Security @ DB/Virtual ServerSharedLow“Friction”/ControlHighSDS V1 targets scenarios that live in the lower left quadrant
  • 6.
    SDS ConceptsProvisioning modelAccount,server, databaseTraditional SQL security modelAuthenticate logins, map to users and rolesAuthorize users and roles to SQL objectsFamiliar SQL programming modelTraditional SQL Server relational modelSupport ADO.Net, ODBCApplications must be partition aware for larger databases
  • 7.
  • 8.
    The Microsoft PlatformStackMobileDesktopServerCloudApplicationDataOS
  • 9.
    Extending SQL DataPlatform to CloudReference DataBusiness IntelligenceData SyncReportingSQL Data Services (SDS)Database-as-a-ServiceSymmetric Programming ModelData Hub AggregationInitial services – core RDBMS capabilities as a service (SDS), Data Sync and Data Hub
  • 10.
  • 11.
    Additional data platformcapabilities as a service: BI/DSS, DW
  • 12.
    New services: ReferenceData, Secure Data Hub
  • 13.
    Enable new usesof data to deliver differentiated business valueService Provisioning ModelEach account has zero or more serversAzure wide, provisioned in a common portalBilling instrumentEach server has one or more databasesContains metadata about the databasesUnit of authenticationUnit of Geo-locationEach database has standard SQL objectsUnit of consistencyContains users, tables, views, indices, etc… Account Server Database
  • 14.
    SDS Server andDatabase Provisioningdemo
  • 15.
    ArchitectureShared SQL infrastructureEachuser database is replicated to multiple serversClient requests are routed to current “primary database”State-of-the-art HA technologyAutomatic failure detection; client request re-routed to new primaryLoad balancing across shared resource poolGateway provides TDS entry point, provisioning capabilityMachine 2Machine 3Machine 4Machine 1SQL InstanceSQL InstanceSQL InstanceSQL InstanceSQL DBSQL DBSQL DBSQL DBUserDB1UserDB3UserDB4UserDB2UserDB1UserDB2UserDB3UserDB4UserDB5UserDB2UserDB3UserDB4UserDB1UserDB5UserDB3UserDB4TDS entry point, provisioningHigh Availability Fabric: Failover and Load balancing
  • 16.
    High Level ArchitectureSQLData Services Gateway TierProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserProtocol ParserManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesManagement ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesPartition ServicesSQL Data Services Back EndSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricDistributed Data FabricMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. ServicesMgmt. Services
  • 17.
    Data Hub “An aggregationof Enterprise, Partner, Desktop, and Device data within SQL Data Services”Windows AzureEnterpriseAzure AppOn-premises AppADO.NET, ODBC, PHP, ...SQL DataServices Data Sync Service SyncEnterpriseSyncSyncUser ManagementDeviceDevice ManagementDesktopClient AppClient AppSync GatewayBusiness Logic / RulesSync ClientSync Client
  • 18.
    Data Hub –Project “Huron”Customer SegmentSmall and medium sized businessesEnterprise – rapid prototype Both Microsoft (v1) and Non-Microsoft device (post v1)Common PatternsMobile device access to corporate dataBusiness to business sharingSecure digital data hub / partner extranets Governmental data (searchable encryption)
  • 19.
    Data Sync ServiceAnalyticalSyncFrameworkTDSReportingSDS ProviderSQL Data ServicesData SyncOnlinePublish APISubscribe APISync API3rd party platformsClient Library APIClient Library APIWindowsDesktops& DevicesSync FrameworkSync FrameworkSync Service ProxySync Service Proxy3rd party DBClient Data ProviderClient Data Provider
  • 20.
    SQL Data ServicesNetwork TopologyApplications use standard SQL client libraries: ODBC, OLEDB, ADO.Net, …ApplicationTDS (tcp:1433)Load balancer forwards ‘sticky’ sessions to TDS protocol tierLoad BalancerTDS (tcp: 1433)GatewayGatewayGatewayGatewayGatewayGatewayData NodeData NodeData NodeData NodeData NodeData NodeTDS (tcp: 1433)Scalability and Availability: Fabric, Failover, Replication and Load balancing
  • 21.
    Scaling Out WithSQL Data ServicesHow can I gain maximum performance from my data tier?What if my application has large data storage requirements?Scaling out with SQL Data Services addresses these requirements
  • 22.
    An Example ScaleOut ArchitectureASP.NetApplicationCustomer 123PartitionAwareApp TierDataPartitions“Shards”
  • 23.
    Software Installation andPatchingSDS ResponsibilitiesSoftware installation, patching and upgradeMinimum interruption to serviceDBA ResponsibilitiesInstall, upgrade your own schemaSupport application and multi-server management model
  • 24.
    Monitoring and ProblemResolutionSDS ResponsibilitiesService health is continuously monitoredHardware, availability, resource usage and etcSelf-healingAutomatic failoverLoad-balancingDedicated MS operation team 24/7Service dashboard for up-to-date health statusDBA ResponsibilitiesRelax and just use the service
  • 25.
    High Availability andDisaster RecoverySDS ResponsibilitiesProvide automatic local HA capabilityPost V1, provide DR capabilityDatabase is automatically backed up with easy recovery for system errorsDBA ResponsibilitiesExport of data to protect against user errors“Clone” a databasePost V1, make available “self-service” restore
  • 26.
    Database Admin Compatibility GoalsSupport common application patterns and TSQL functionalitySupport common DBA tasksFocus on logical data administrationPhysical administration provided by service
  • 27.
    Logical AdministrationDatabase SchemaDesignDatabase User and Permission ManagementIndex ManagementQuery TuningStatistics ManagementData Import/ExportReporting…DBA knowledge is compatible with on-premise SQL
  • 28.
    Logical vs. PhysicalAdministrationSupportedCreate/Drop databaseCreate/Drop tableCreate/Drop userIndex rebuildUpdate statistics…Not SupportedDatabase file placementFilegroup managementServer configuration optionsMirroring…
  • 29.
    Example: CREATE TABLECREATETABLE [[[ database_name . ] schema_name . table_name ( { <column_definition> | <computed_column_definition> | <column_set_definition> } [ < table_constraint> ] [ ,...n ] ) [ ON { partition_scheme_name … | filegroup | "default" } ] [ TEXTIMAGE_ON … ] [WITH ( < table_option > [ ,...n ] ) ]…No support for placement ex: FILEGROUP, TEXTIMAGE
  • 30.
    Performance TroubleshootingDetect blockingquery and deadlocksQuery optimizationTop N expensive queriesExcessive recompile, bad query planIndex tuningSome of existing SQL facilities will continue to workSET SHOWPLANSET STATISTICS DMV/DMF like system views…