Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.1
Outline
Introduction
What is a distributed DBMS
Problems
Current state-of-affairs
Background
Distributed DBMS Architecture
Distributed Database Design
Semantic Data Control
Distributed Query Processing
Distributed Transaction Management
Parallel Database Systems
Distributed Object DBMS
Database Interoperability
Current Issues
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.2
File Systems
program 1
data description 1
program 2
data description 2
program 3
data description 3
File 1
File 2
File 3
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.3
Database Management
database
DBMS
Application
program 1
(with data
semantics)
Application
program 2
(with data
semantics)
Application
program 3
(with data
semantics)
description
manipulation
control
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.4
Motivation
Database
Technology
Computer
Networks
integration distribution
integration
integration ≠ centralization
Distributed
Database
Systems
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.5
Distributed Computing
A concept in search of a definition and a name.
A number of autonomous processing elements
(not necessarily homogeneous) that are
interconnected by a computer network and that
cooperate in performing their assigned tasks.
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.6
Synonymous terms
distributed function
distributed data processing
multiprocessors/multicomputers
satellite processing
backend processing
dedicated/special purpose computers
timeshared systems
functionally modular systems
Distributed Computing
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.7
Processing logic
Functions
Data
Control
What is distributed …
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.8
What is a Distributed Database
System?
A distributed database (DDB) is a collection of multiple,
logically interrelated databases distributed over a
computer network.
A distributed database management system (D–DBMS)
is the software that manages the DDB and provides an
access mechanism that makes this distribution
transparent to the users.
Distributed database system (DDBS) = DDB + D–DBMS
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.9
A timesharing computer system
A loosely or tightly coupled multiprocessor
system
A database system which resides at one of the
nodes of a network of computers - this is a
centralized database on a network node
What is not a DDBS?
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.10
Centralized DBMS on a Network
Site 5
Site 1
Site 2
Site 3Site 4
Communication
Network
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.11
Distributed DBMS Environment
Site 5
Site 1
Site 2
Site 3Site 4
Communication
Network
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.12
Implicit Assumptions
Data stored at a number of sites each site
logically consists of a single processor.
Processors at different sites are interconnected
by a computer network no multiprocessors
parallel database systems
Distributed database is a database, not a
collection of files data logically related as
exhibited in the users’ access patterns
relational data model
D-DBMS is a full-fledged DBMS
not remote file system, not a TP system
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.13
Shared-Memory Architecture
Examples : symmetric multiprocessors (Sequent,
Encore) and some mainframes
(IBM3090, Bull's DPS8)
P1 Pn
M
D
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.14
Shared-Disk Architecture
Examples : DEC's VAXcluster, IBM's IMS/VS
Data Sharing
D
P1
M1
Pn
Mn
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.15
Shared-Nothing Architecture
Examples : Teradata's DBC, Tandem, Intel's
Paragon, NCR's 3600 and 3700
P1
M1
D1
Pn
Mn
Dn
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.16
Manufacturing - especially multi-plant
manufacturing
Military command and control
EFT
Corporate MIS
Airlines
Hotel chains
Any organization which has a
decentralized organization structure
Applications
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.17
Distributed DBMS Promises
Transparent management of distributed,
fragmented, and replicated data
Improved reliability/availability through
distributed transactions
Improved performance
Easier and more economical system expansion
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.18
Transparency
Transparency is the separation of the higher level
semantics of a system from the lower level
implementation issues.
Fundamental issue is to provide
data independence
in the distributed environment
Network (distribution) transparency
Replication transparency
Fragmentation transparency
horizontal fragmentation: selection
vertical fragmentation: projection
hybrid
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.19
Example
TITLE SAL
PAY
Elect. Eng. 40000
Syst. Anal. 34000
Mech. Eng. 27000
Programmer 24000
PROJ
PNO PNAME BUDGET
ENO ENAME TITLE
E1 J. Doe Elect. Eng.
E2 M. Smith Syst. Anal.
E3 A. Lee Mech. Eng.
E4 J. Miller Programmer
E5 B. Casey Syst. Anal.
E6 L. Chu Elect. Eng.
E7 R. Davis Mech. Eng.
E8 J. Jones Syst. Anal.
EMP
ENO PNO RESP
E1 P1 Manager 12
DUR
E2 P1 Analyst 24
E2 P2 Analyst 6
E3 P3 Consultant 10
E3 P4 Engineer 48
E4 P2 Programmer 18
E5 P2 Manager 24
E6 P4 Manager 48
E7 P3 Engineer 36
E8 P3 Manager 40
ASG
P1 Instrumentation 150000
P3 CAD/CAM 250000
P2 Database Develop. 135000
P4 Maintenance 310000
E7 P5 Engineer 23
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.20
Transparent Access
SELECT ENAME,SAL
FROM EMP,ASG,PAY
WHERE DUR > 12
AND EMP.ENO = ASG.ENO
AND PAY.TITLE = EMP.TITLE Paris projects
Paris employees
Paris assignments
Boston employees
Montreal projects
Paris projects
New York projects
with budget > 200000
Montreal employees
Montreal assignments
Boston
Communication
Network
Montreal
Paris
New
York
Boston projects
Boston employees
Boston assignments
Boston projects
New York employees
New York projects
New York assignments
Tokyo
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.21
Distributed Database - User View
Distributed Database
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.22
Distributed DBMS - Reality
Communication
Subsystem
User
Query
DBMS
Software
DBMS
Software
User
Application
DBMS
Software
User
ApplicationUser
Query
DBMS
Software
User
Query
DBMS
Software
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.23
Potentially Improved Performance
Proximity of data to its points of use
Requires some support for fragmentation and replication
Parallelism in execution
Inter-query parallelism
Intra-query parallelism
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.24
Parallelism Requirements
Have as much of the data required by each
application at the site where the application
executes
Full replication
How about updates?
Updates to replicated data requires implementation of
distributed concurrency control and commit protocols
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.25
System Expansion
Issue is database scaling
Emergence of microprocessor and workstation
technologies
Demise of Grosh's law
Client-server model of computing
Data communication cost vs telecommunication
cost
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.26
Distributed DBMS Issues
Distributed Database Design
how to distribute the database
replicated & non-replicated database distribution
a related problem in directory management
Query Processing
convert user transactions to data manipulation
instructions
optimization problem
min{cost = data transmission + local processing}
general formulation is NP-hard
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.27
Distributed DBMS Issues
Concurrency Control
synchronization of concurrent accesses
consistency and isolation of transactions' effects
deadlock management
Reliability
how to make the system resilient to failures
atomicity and durability
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.28
Directory
Management
Relationship Between
Issues
Reliability
Deadlock
Management
Query
Processing
Concurrency
Control
Distribution
Design
Distributed DBMS © 2001 M. Tamer Özsu & Patrick Valduriez Page 1.29
Operating System Support
operating system with proper support for database
operations
dichotomy between general purpose processing
requirements and database processing requirements
Open Systems and Interoperability
Distributed Multidatabase Systems
More probable scenario
Parallel issues
Related Issues

Intro to Distributed Database Management System

  • 1.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.1 Outline Introduction What is a distributed DBMS Problems Current state-of-affairs Background Distributed DBMS Architecture Distributed Database Design Semantic Data Control Distributed Query Processing Distributed Transaction Management Parallel Database Systems Distributed Object DBMS Database Interoperability Current Issues
  • 2.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.2 File Systems program 1 data description 1 program 2 data description 2 program 3 data description 3 File 1 File 2 File 3
  • 3.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.3 Database Management database DBMS Application program 1 (with data semantics) Application program 2 (with data semantics) Application program 3 (with data semantics) description manipulation control
  • 4.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.4 Motivation Database Technology Computer Networks integration distribution integration integration ≠ centralization Distributed Database Systems
  • 5.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.5 Distributed Computing A concept in search of a definition and a name. A number of autonomous processing elements (not necessarily homogeneous) that are interconnected by a computer network and that cooperate in performing their assigned tasks.
  • 6.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.6 Synonymous terms distributed function distributed data processing multiprocessors/multicomputers satellite processing backend processing dedicated/special purpose computers timeshared systems functionally modular systems Distributed Computing
  • 7.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.7 Processing logic Functions Data Control What is distributed …
  • 8.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.8 What is a Distributed Database System? A distributed database (DDB) is a collection of multiple, logically interrelated databases distributed over a computer network. A distributed database management system (D–DBMS) is the software that manages the DDB and provides an access mechanism that makes this distribution transparent to the users. Distributed database system (DDBS) = DDB + D–DBMS
  • 9.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.9 A timesharing computer system A loosely or tightly coupled multiprocessor system A database system which resides at one of the nodes of a network of computers - this is a centralized database on a network node What is not a DDBS?
  • 10.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.10 Centralized DBMS on a Network Site 5 Site 1 Site 2 Site 3Site 4 Communication Network
  • 11.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.11 Distributed DBMS Environment Site 5 Site 1 Site 2 Site 3Site 4 Communication Network
  • 12.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.12 Implicit Assumptions Data stored at a number of sites each site logically consists of a single processor. Processors at different sites are interconnected by a computer network no multiprocessors parallel database systems Distributed database is a database, not a collection of files data logically related as exhibited in the users’ access patterns relational data model D-DBMS is a full-fledged DBMS not remote file system, not a TP system
  • 13.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.13 Shared-Memory Architecture Examples : symmetric multiprocessors (Sequent, Encore) and some mainframes (IBM3090, Bull's DPS8) P1 Pn M D
  • 14.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.14 Shared-Disk Architecture Examples : DEC's VAXcluster, IBM's IMS/VS Data Sharing D P1 M1 Pn Mn
  • 15.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.15 Shared-Nothing Architecture Examples : Teradata's DBC, Tandem, Intel's Paragon, NCR's 3600 and 3700 P1 M1 D1 Pn Mn Dn
  • 16.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.16 Manufacturing - especially multi-plant manufacturing Military command and control EFT Corporate MIS Airlines Hotel chains Any organization which has a decentralized organization structure Applications
  • 17.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.17 Distributed DBMS Promises Transparent management of distributed, fragmented, and replicated data Improved reliability/availability through distributed transactions Improved performance Easier and more economical system expansion
  • 18.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.18 Transparency Transparency is the separation of the higher level semantics of a system from the lower level implementation issues. Fundamental issue is to provide data independence in the distributed environment Network (distribution) transparency Replication transparency Fragmentation transparency horizontal fragmentation: selection vertical fragmentation: projection hybrid
  • 19.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.19 Example TITLE SAL PAY Elect. Eng. 40000 Syst. Anal. 34000 Mech. Eng. 27000 Programmer 24000 PROJ PNO PNAME BUDGET ENO ENAME TITLE E1 J. Doe Elect. Eng. E2 M. Smith Syst. Anal. E3 A. Lee Mech. Eng. E4 J. Miller Programmer E5 B. Casey Syst. Anal. E6 L. Chu Elect. Eng. E7 R. Davis Mech. Eng. E8 J. Jones Syst. Anal. EMP ENO PNO RESP E1 P1 Manager 12 DUR E2 P1 Analyst 24 E2 P2 Analyst 6 E3 P3 Consultant 10 E3 P4 Engineer 48 E4 P2 Programmer 18 E5 P2 Manager 24 E6 P4 Manager 48 E7 P3 Engineer 36 E8 P3 Manager 40 ASG P1 Instrumentation 150000 P3 CAD/CAM 250000 P2 Database Develop. 135000 P4 Maintenance 310000 E7 P5 Engineer 23
  • 20.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.20 Transparent Access SELECT ENAME,SAL FROM EMP,ASG,PAY WHERE DUR > 12 AND EMP.ENO = ASG.ENO AND PAY.TITLE = EMP.TITLE Paris projects Paris employees Paris assignments Boston employees Montreal projects Paris projects New York projects with budget > 200000 Montreal employees Montreal assignments Boston Communication Network Montreal Paris New York Boston projects Boston employees Boston assignments Boston projects New York employees New York projects New York assignments Tokyo
  • 21.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.21 Distributed Database - User View Distributed Database
  • 22.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.22 Distributed DBMS - Reality Communication Subsystem User Query DBMS Software DBMS Software User Application DBMS Software User ApplicationUser Query DBMS Software User Query DBMS Software
  • 23.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.23 Potentially Improved Performance Proximity of data to its points of use Requires some support for fragmentation and replication Parallelism in execution Inter-query parallelism Intra-query parallelism
  • 24.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.24 Parallelism Requirements Have as much of the data required by each application at the site where the application executes Full replication How about updates? Updates to replicated data requires implementation of distributed concurrency control and commit protocols
  • 25.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.25 System Expansion Issue is database scaling Emergence of microprocessor and workstation technologies Demise of Grosh's law Client-server model of computing Data communication cost vs telecommunication cost
  • 26.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.26 Distributed DBMS Issues Distributed Database Design how to distribute the database replicated & non-replicated database distribution a related problem in directory management Query Processing convert user transactions to data manipulation instructions optimization problem min{cost = data transmission + local processing} general formulation is NP-hard
  • 27.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.27 Distributed DBMS Issues Concurrency Control synchronization of concurrent accesses consistency and isolation of transactions' effects deadlock management Reliability how to make the system resilient to failures atomicity and durability
  • 28.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.28 Directory Management Relationship Between Issues Reliability Deadlock Management Query Processing Concurrency Control Distribution Design
  • 29.
    Distributed DBMS ©2001 M. Tamer Özsu & Patrick Valduriez Page 1.29 Operating System Support operating system with proper support for database operations dichotomy between general purpose processing requirements and database processing requirements Open Systems and Interoperability Distributed Multidatabase Systems More probable scenario Parallel issues Related Issues