Microsoft SQL Server Distributing Data with R2 Bertucci

Distributing Data Techniques with Distributing Data Techniques with Distributing Data Techniques with SQL Server 2008 R2 Q Speaker: Paul Bertucci Founder, Database Architechs d b hi h Silicon Valley SQL Server User Group October 19, 2010 Mark Ginnebaugh, User Group Leader, mark@designmind.com

Agenda g • Determining where data is needed • What SQL Server options do you have • How to get it there • Work within your constraints  Security  Regulatory (like PII…)  Administratively Ad i i t ti l  Data volatility/sizes  SLA’s you must deliver  Disaster Recovery/Business Continuity  Skills/Knowledge of your team  Resources (like $$’s, so on)

Reasons to Distribute Data Reasons to Distribute Data

SQL Server 2008 SQL Server 2008 Reporting/ODS Primary Reporting OLTP Server 19FIG02 OLTP DB Rpt DB pt SQL Server 2008 Enabling/Partitioning Europe Server SQL Server 2008 USA xyz DB SQL Server 2008 (Headquarters) Asia xyz DB Server xyz DB SQL Server 2008 SQL Server 2008 North America Regionalization Europe Region Region xyz DB y (multiple owners) xyz DB y SQL Server 2008 SQL Server 2008 Primary Failover Hot Spare (Fail-over) xyz DB xyz DB

Distributing periodic updates to Data Marts from a “master” Data Warehouse Data Mart Master SQL Server Data Warehouse 2008 Data Mart Data Mart SQL Server 2000 SQL Server 2005

Enabling a Data Warehouse from Legacy Systems ORACLE Master Data Warehouse SQL Server 2005 SQL Server 2008

Initialization of tables Heterogeneous for a heterogeneous Subscriber Subscriber in Replication ORACLE Publisher Subscriber SQL Server 2008 SQL Server 2008 Subscriber SQL Server 2005

Pulling in Sales Order information Sales Force Automation from disparate sources Application for import into SQL Server 2008 Access SQL Server Excel 2008

Data Mart OLTP Database SQL Server 2008 SQL Server 2008 Excel Export from the OLTP Database to a business intelligence Data Mart & Excel

Connection Results Results SQL Server SQL Server 2008 2008 Client Local Server Remote Server

Decision‐Tree Path Traversal Database Mirroring Integration I t ti With Snapshots S h t Services 1a2c3 1a2d3 Detach/ 1a2e3 Attach BCP 1a2b3 b c d a 1a2 e 1a2a3 1 2 3 a b 1b2 1e2 Log Sync e 1 c d Shipping Framework 1c2 1d2 Backup/ Data Restore Replication Linked Servers

Data Log Database Mirroring Linked Backup/ Detach/ Sync SSIS BCP Replication Shipping with Snapshots Servers Restore Attach Framework Large Data High Volatility On Line Many Targets Low Latency Transformation/ Filtering

Data Replication Data Replication

Central Publisher SQL Server SQL Server 2008 (default option) 2005 Adventure Adventure Works2008 Works2008 translog SQL Server translog Subscription 2008 Subscription Server 19FIG10 Server Adventure Works2008 translog Publication Server distribution SQL Server 2000 Oracle Distribution Server Adventure Adventure Works2008 Works2008 translog Subscription Subscription Server Server 15

Central Publisher SQL Server 19FIG11 Remote Distributor SQL Server 2008 2005 Adventure Adventure Works2008 Works2008 SQL Server translog 2008 translog Subscription Subscription Server Adventure Server Works2008 translog Publication Server SQL Server 2008 SQL Server 2000 Oracle distribution Remote Distribution Adventure Adventure Works2008 Server Works2008 translog Subscription Subscription Server Server

Publishing Subscriber SQL Server SQL Server 2008 2005 SQL Server Publication 2008 Adventure Adventure Server Works2008 Works2008 Adventure 19FIG12 Works2008 translog S translog Subscription translog Subscription Distribution Server Server Server distribution Subscription SQL Server Serveer 2008 Adventure SQL Server Works2008 2000 tion Oracle er translog Publicat Serve Adventure Adventure Works2008 Works2008 translog Subscription ion Subscription Distributi Server r Server Server distribution 17

Publication SQL Server SQL Server tion 2008 2008 19FIG13 er er Central Serve Publicat Serve Adventure Adventure Works2008 Works2008 translog Subscriber translog ion Distribution Distributi Server r Server distribution distribution SQL Server 2008 Subscription Server Adventure Works2008 SQL Server SQL Server Publication ation 2008 2008 ver ver Publica Serv Serv Adventure Adventure Works2008 Works2008 translog translog ution ution er er Serve Serve Distribu Distribu distribution distribution

Multiple Publishers of a Single Table 19FIG14Server on on Subscriptio Subscriptio SQL S SQL Server S Server Server 2008 2008 Adventure Adventure on on Works2008 Works2008 Publicatio Publicatio Serverr Server translog translog Distribution Distribution Server Server distribution distribution Territory 1 Territory 2 Customers Table CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate 1345 1 AW1345 I X69G9.. 120203 1356 2 AW1356 I W211G.. 051605 2354 1 AW2354 S 7SQ78K.. 106705 3346 2 AW3346 I W12DV.. 022305 7643 2 AW7643 S WZ8R4.. 122205 7901 2 AW7901 I S2345X.. 041506 8921 1 AW8921 I RT66Y.. 031206

SQL Server Updating U d ti blication 2008 Subscriber Server 19FIG15 Adventure Works2008 Pub S translog SQL Server Queue stribution (Queue Reader Agent) Server S distribution Dis Update Customers set AccountNumber = null where CustomerID = 12345 SQL Server SQL Server 2008 2008 Adventure Adventure Works2008 Works2008 W k 2008 translog translog Subscription Subscription Server Server

SQL Server SQL Server 2008 2008 19FIG16 Adventure Works2008 Peer-to-Peer Adventure Works2008 translog translog Publication Publication Server Server North American o e ca Asia sa SQL Server 2008 Server Server distribution Peer Originator id: 1 Distribution Server Peer Originator id: 2 Customers Table CustomerID C t ID TerritoryID T it ID AccountNumber C A tN b CustomerType T rowguid id ModifiedDate M difi dD t 1345 1 AW1345 I X69G9.. 120203 1356 2 AW1356 I W211G.. 051605 2354 1 AW2354 S 7SQ78K.. 106705 3346 2 AW3346 I W12DV.. W12DV 022305 7643 3 AW7643 S WZ8R4.. 122205 7901 5 AW7901 I S2345X.. 041506 8921 4 AW8921 I RT66Y.. 031206

Publisher Publication P bli ti AW_Vertical Adventure Works2008 translog CustomerV (Article)19FIG05 SQL Server 2008 CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate CustomerID TerritoryID CustomerType 1345 1 AW1345 I X69G9.. 120203 1345 1 I 1356 2 AW1356 I W211G.. 051605 1356 2 I 2354 1 AW2354 S 7SQ78K.. 106705 2354 1 S 3346 2 AW3346 I W12DV.. 022305 3346 2 I 7643 3 AW7643 S WZ8R4.. 122205 7643 3 S 7901 5 AW7901 I S2345X.. 041506 7901 5 I 8921 4 AW8921 I RT66Y.. 031206 8921 4 I

Publisher Publication AW_Horizontal Adventure Works2008 19FIG06 CustomerH (Article) translog SQL Server 2008 CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate 1345 1 AW1345 I X69G9.. 120203 1356 2 AW1356 I W211G.. 051605 2354 1 AW2354 S 7SQ78K.. 106705 3346 2 AW3346 I W12DV.. 022305 Only these Rows! 7643 3 AW7643 S WZ8R4.. 122205 7901 5 AW7901 I S2345X.. 041506 8921 4 AW8921 I RT66Y.. 031206 CustomerID TerritoryID AccountNumber CustomerType yp rowguid g ModifiedDate 2354 1 AW2354 S 7SQ78K.. 106705 7643 3 AW7643 S WZ8R4.. 122205

Publisher Publication Adventure Ad t 19FIG07 AW_H_and_V Works2008 07FIG08 translog CustomerHV (Article) SQL Server 2008 CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate 1345 1 AW1345 I X69G9.. 120203 1356 2 AW1356 I W211G.. 051605 Only these Columns 2354 1 AW2354 S 7SQ78K.. 106705 and 3346 2 AW3346 I W12DV.. 022305 these Rows! 7643 3 AW7643 S WZ8R4.. 122205 7901 5 AW7901 I S2345X.. 041506 8921 4 AW8921 I RT66Y.. 031206 CustomerID TerritoryID y CustomerType 2354 1 S 7643 3 S

AW_H_and_J (Publication) CustomerH (Article) 19FIG08 07FIG09 OrdersHJ (Article) CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate Only the orders 1345 1 AW1345 I X69G9.. X69G9 120203 for 1356 2 AW1356 I W211G.. 051605 customer 2354 1 AW2354 S 7SQ78K.. 106705 CustomerID 2354 type “S” 3346 2 AW3346 I W12DV.. 022305 7643 3 AW7643 S WZ8R4.. WZ8R4 122205 7643 7901 5 AW7901 I S2345X.. 041506 8921 4 AW8921 I RT66Y.. 031206 join SalesOrderID RevisionNumber OrderDate DueDate Status CustomerID 43659 1 7/1/2001 7/13/2001 5 1345 43660 1 7/1/2001 7/13/2001 5 1356 43661 1 7/1/2001 7/13/2001 5 2354 SalesOrderID OrderDate CustomerID 43662 2 7/1/2001 7/13/2001 5 3346 43661 7/1/2001 2354 43663 1 7/1/2001 7/13/2001 5 7643 43664 1 7/1/2001 7/13/2001 5 7901 43663 7/1/2001 7643 43665 1 7/1/2001 7/13/2001 5 8921 25

Publisher AW (Publication) (P bli ti ) Subscriber SQL Server cation 19FIG17 2008 Customers (Article) rver SQL Server Public Adventure 2008 Ser Works2008 Adventure translog Works2008 bution PUSH rver translog Distrib Ser distribution di t ib ti Subscription Server PUSH: initiated by Publisher PULL (agent will be on the Publisher if local distribution, or on the Distributor if remote distribution) PULL: initiated by Subscriber y (agent will be on the subscriber)

Data Access Latency Distributing Data Distributing Data Autonomy Sites (locations) Frequency Network Machines Owner Other REPLICATION Read Only Each site only Central Publisher fast/ n Transactional repl Reporting short high many high 1 OLTP needs regional stable server/site site data filter by region Read Only Each site only Central Publisher fast/ n Snapshot repl Reporting long high many low 1 OLTP needs regional stable server/site site data filter by region Read Mostly ead ost y Regional updates Central Publisher g p fast/ n Transactional repl A few updates short high < 10 medium 1 OLTP on one table stable server/site site Updating Subs Read Mostly Regional update Central Publisher slow/ n All Merge repl A few updates medium high < 10 medium all tables unreliab server/site update Read equal Regional update Peer-to-Peer fast/ 1 All Transactional Equal updates short high < 10 medium all tables stable server/site update repl Inserts 1 Each site only Central Subscriber fast/ 1 Transactional repl (new orders) short high many high needs regional stable server/site report data site Central Publisher Hot/Warm / Very high <2 high fast/ f / 1 Fail-over Remote Distributor 1 OLTP Spare short stable server/site site Transactional repl

Log Shipping Yes, it is still here.. ,

Log Shipping “Source” SQL Server 2008 “Destination” SQL Server 2008 Q TxnLog T L Primary CallOne DB backups Secondary Server Server translog CallOne DB BackupCallOne_tlog_200405141120.TRN pped wer LogShareCallOne_tlog_200405141120.TRN LogShareCallOne tlog 200405141120 TRN TxnLog Last log ship Delay Answ TxnLog T L Copies Restores “Monitor” SQL Server 2008 S Delay between logs loaded Delay Answer Monitor Server MSDB DB Don’t build on this, it is being deprecated !

Database Mirroring with Snapshots with Snapshots

Client Client Client Client Network A SQL Server 2008 SQL Server 2008 Mirror Server Mi S DB Mirroring Adventure D Principal Adventure D Works DB Server B Works DB translog t l translog C SQL Server xyz Witness t ess Server MSDB DB 31

SQL Server 2008 SQL Server 2008 Mirror Server Principal 10 Server AdventureWorks AdventureWorks DB DB translog translog Database Database Snapshot Snapshot comes the mirror. If this server node SQL Server 2008 m n Witness MSDB DB Server bec t 32

SQL Server 2008 01AdventureWork Source CREATE DB ‘xyz’ s DB AS SNAPSHOT OF Server AdventureWorks translog Database D t b Snapshot --- Reverting --- if needed --- RESTORE DB AdventureWorks FROM DATABASE_SNAPSHOT = ‘xyz’

Source Data Pages Source 02 SQL Server 2008 SQL Sparse Server File Pages Snapshot AdventureWorks AdventureWorks DB DB Empty Sparse file of a Snapshot just created (no p ( updates to original g data pages have occurred yet) 34

Source Data Pages Source 03 SQL Server 2008 SQL Sparse Server File Pages Snapshot AdventureWorks AdventureWorks DB DB System Catalog Copy of original of changed pages pages for snapshot only when a page is changed (Copy-on-write) 35

Snapshot Source Data Users Pages SELECT …..data……. FROM AdventureWorks SNAPSHOT 04 SQL Server 2008 SQL Server Snapshot AdventureWorks AdventureWorks DB DB System Catalog Sparse of changed pages File Pages 36

SQL Server 2008 SQL Server 2008 Mirror Server Principal 10 Server AdventureWorks AdventureWorks DB DB translog translog Database Database Snapshot Snapshot comes the mirror. If this server node SQL Server 2008 m n Witness MSDB DB Server bec t 37

PH Topology With Snapshots Real time Reporting R ti Users Network SQL Server 2008 N Principal Server SQL Server 2008 OLTP Application n Active Mirror Server stered Replication Adventure Adventure Works DB Works DB Clus translog translog Database Passive PH Snapshot Topology Principal Network Server SQL Server 2008 Point in time 38 Reporting Users

Publisher DB Mirror and SQL Server 2008 SQL Server 2008 Replication Principal Mirror SQL Server 2008 Server Server Subscriber Distributor s bu o SQL Server 2008 SQL Server 2008 Principal Mirror Server Server SQL Server 2008 SQL Server 2008 Witness Server Subscriber 39

Integration Services

Excel OLAP Data Warehouse Access Multidimensional OLTP Cube Databases Other Data Sources OLTP Databases SQL Server Flat XML Web Services files

Package P k Task Task Task ainer Source Cache e Conta Data Flow Transformation Task Destination D ti ti Task Task Task

SSIS Data Mart Join and Aggregate e OLTP Database Tas to SQL Server sk AdventureWorks 2008 d UnleashedDataMart SQL Server 2008 Excel .csv flat file Export from the OLTP Database to a business intelligence Data Mart & Excel/.csv flat file

Linked Servers

OLE DB Provider for MS SQL Server SQL Server 6.5 – 2008 OLE DB Provider DLL’s OLE DB Provider Sybase for ODBC Informix DB2 ODBC Others OLE DB Provider Excel spreadsheet for Jet Jet FoxPro Database database Dbase file MS OLE DB OLE DB Provider file Access Database E Provider P id for Oracle f O l SQL Server ORACLE 2008 OLE DB Provider P id for Indexing Service Active Directory Services Index Server Others Client Tier Server Tier Remote Data Source Tier (can be “Linked”)

Might be to easy? Might be to easy? ------------------------------------------------------------- -- Selecting data from Excel spreadsheet via Linked Server -- ------------------------------------------------------------- SELECT substring(CustomerID,1,5) as 'CustID', substring(ContactName,1,18) as ContactName, substring(City,1,10) as 'Cit ' substring(Country,1,15) as 'C b t i (Cit 1 10) 'City', b t i (C t 1 15) 'Country'' t FROM [Linked ExcelSW]...[SWCustomers$] ORDER BY ContactName g go CustID ContactName City Country ------ ------------------ ---------- --------------- PICCO Adam Greifer Los Angele USA BLAUS John Monroe Concord USA CHOPS Martin Sommer Berkeley USA

Microsoft Sync Framework Microsoft Sync Framework A comprehensive synchronization platform that enables collaboration and offline access for enables collaboration and offline access for applications, services, and devices. It features technologies and tools that enable roaming, sharing, and taking data offline. By using Sync Framework, developers can build B i S F k d l b ild sync ecosystems that integrate any application with any data from any store that uses any y y y protocol over any network. Mentioned here because of its replication‐like behavior for “occasionally connected” applications. occasionally connected applications

Others • BCP • Detach/Attach • Backup/Restore k /

Copy on Write Technology

DB Mirroring Client Client Client Client Network A SQL Server 2008 SQL Server 2008 Mirror Server Mi S Principal Adventure D Adventure Works DB D Server B Works DB translog t l translog C SQL Server xyz Witness t ess Server MSDB DB 50

Source Data Pages Source 03 SQL Server 2008 SQL Sparse Server File Pages Snapshot AdventureWorks AdventureWorks DB DB System Catalog Copy of original of changed pages pages for snapshot only when a page is changed (Copy-on-write) 51

Database Architechs Paul Bertucci  Founder Database Architechs  Former Chief Data Architect – Veritas & Symantec  Currently C rrentl Chief Architect - A todesk Autodesk  Author of SQL Server 2000, 2005 & 2008/R2 Unleashed  Co- Co-Author of ADO.NET in 24 hours  Author MS SQL Server High Availability  Author Sybase Performance & Tuning  Author Sybase Physical DB Design  Author Chen Entity/Relationship Modeling  Veritas SQL Server Performance Series pbertucci@dbarchitechs.com

To learn more or inquire about speaking opportunities, please contact: Mark Ginnebaugh, User Group Leader Mark Ginnebaugh User Group Leader mark@designmind.com

http://www.designmind.com	874
http://markginnebaugh.com	9
http://webcache.googleusercontent.com	2
http://sps2010.designmind.com	2

Microsoft SQL Server Distributing Data with R2 Bertucci

by Mark Ginnebaugh on Oct 25, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 887

Statistics

Microsoft SQL Server Distributing Data with R2 Bertucci — Presentation Transcript