Cdc

UBSS
UNIX Based System Software

CDC
Consistent Datastore Copies

MCC/CPC/CIS

CDC Model

Network Shared Memory with Update


Notifications
 Simple abstraction already understoods by

programmers
 Natural transistion from sequential to distributed

applications
 Shared Memory Persistance

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 2

CDC Benefits

Applications loosely coupled


Replicated Data Services for Fault

tolerant applications

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 3

ATC OPERATOR Requirements

ATC controllers require :


Rapid response time


and
Fixed response time


to commands and commands acknowledgment

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 4

ATC Centers Physical Lay Out

FDP
FDP
Request
and
Ack

Operator Operator Operator Operator Operator
Position Position Position Position Position

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 5

Push or Pull model

Pull model


clients request information from the server when


needed
Push model


servers push the information to clients before


needed

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 6

Push model

FDP
FDP


01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 7

Push model and data replication

Push model implies : data replication
•

Data replication implies data consistency
•
between copies :
Strong consistency


Or
Weak consistency


01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 8

Strong consitency
Strong consistency implies :


Atomicity : all receive the message or no one


Causality : all messages are received by all in


the same order

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 9

Atomicity
A B C
Data Update

Ack

Group Ack

Update Available

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 10

Causality (case study)
A B C
non causal event = same time Data Update
Data Update

Ack

Ack

Ordering Message (Atomic)

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 11

CDC lightweight protocol

CDC provids weak consistency :


Replicated information may temporarily differ while data copies


are updated = data are always available
Serial consistency on the same record is guaranteed = an old


data will never replace a newer data, but data update may be
lost
Periodic consistency check = One global acknowledgment at


periodic time (master/slave configuration)

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 12

Temporary inconsistency
Data Update N N N
= Version N

N+1 N N

N+1 N+1 N

N+1 N+1 N+1
= Version N +1

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 13

Periodic Consistency checking
A B C D E

A B C D E
T
Update
CAM
T
T Update

CAM
Update
T

CAM
CAM

T

CAM
Master
01/04/2
RRM : Retransmission Request Message (CAM : Central Announcement Message) Version 6

MCC/CPC/CIS
UBSS/CDC 14

CDC algorithms

replication provides :
Full

 Multiple Reader Single Writer : MRSW (read
replication consistency).
 Multiple Readers Multiple Writers : MRMW
(write shared consistency).

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 15

CDC Write Shared Consistency Semantic

Shared : Write Shared Data are those that are
Write
frequently and concurrently writen by multiple
threads without intervening synchronization.
Concurreny writes are safe, because the programmers
knows that each thread updates separate portions of
the Data.
Because of the way the data are laid out in memory,
false sharing may be present.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 16

Write Sharing Model

Array sharing, Unshared record :

notification false shared : filtering.


Shared record, unshared fields :

 false shared : filtering.

Shared fields :

real shared : Application synchronisation

(Transactionnal).

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 17

Write Shared : Array False sharing

CDC shared record array

Unshared Record

thread 1

thread 2

thread 3

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 18

Write Shared : Record False sharing => Colision

thread 1 thread 2 thread 3

Shared Record

Unshared fields

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 19

CDC shared memory access semantic

record #)
READ(data,

WRITE(data, record #)

SUBSCRIBE

(record #)
NOTIFY

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 20

Consistent Datastore Copies (CDC) : SCOPE 1/2

Distributed algorithm for managing replicated information with stringent response

time (access) requirements on a large number of nodes
Replicated information may temporarily differ while data copies are updated. The

copies converge when data modification ceases.
The algorithm is fail-safe against node failures and insertions.


A predefined node is the master and provides periodic consistency checking and

recovery.
In case of failure, a new master is automatically allocated by means of priority.


Application process can subscribe to datastore. It will then be informed about any

changes of the datastore.
User has no knowledge about other producers or consumers.


Serial consistency on the same record is guaranteed.


01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 21

Consistent Datastore Copies (CDC) : SCOPE 2/2

A datastore copy is a table of records


Sorting algorithm with application defined key


A datastore copy has fixed known attribute (size of record, number of records,

name and identification of the datastore).
All datastore copy with the same name will contain the same informations on all

nodes.The application may claim access to the datastore copy.
The lock is node wide. (It is possible to group the operations).


Access time must be limited. Application must take care of deadlocks.


Application defined automatic conversion function call backfor read and write


Four different ways to access store records :

Physical access using pointers


Sequential access


Direct access


Indexed sequential access (Each user process can provide a compare function of

key to built a table of indexes)
01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 22

CDC : Distributed Shared Memory Model

 Structure
:
 Layout of the shared data memory :

Array of C or Ada record data type


 Granularity:
 Size of the unit of sharing

 C or Ada strutures size, which can vary to match
the application requirements

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 23

CDC MECHANISMS Master
Slave

Slave

Appli
Appli Appli

Data Store
Data Store Data Store

data bus

LAN
Slave
Slave

Data Store Data Store

Appli Appli Appli

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 24

UBSS CDC : System Design
Radar

RFP RDP FDP

Radar1 Radar2 Radar3 Radar4 System Tracks Identification Flight Plan

MMI
MMI
MMI
MMI
01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 25

CDC : Context diagram

Event
Application Notification

Library
Library Interface Message
Interface Interface
CDC

CDC protocol

CDC store collision store
LAN

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 26

CDC : distribution and processing 1/3

For each record, CDC maintains two field :

- the Record Update Counter (RUC); identifies the version of the record. It is
incremented by one for each modification.
- the Record Update Originator (RUO); identifies the node where the last update has
been initiated.
A user gains exclusive access to the local datastore copy and update one or several

records. RUC and RUO are also updated by CDC.
When exclusive access is released, the updated records are packed into one or more

Update Initiation Messages (UIMs) and distributed to other nodes on the LAN. UDP
datagram broadcast facility is used.
Each receiver node (master and slave) applies the same algorithm to update their data

store.
For each record received in UIM message, if the RUC(UIM) > RUC (datastore), it

replaces the datastore record
If RUC(UIM) = RUC (datastore) and RUO(UIM) /= RUO (datastore) then a collision

occured
RRM : Retransmission Request Message 01/04/2
UIM : Update Initiation Messages Version 6

MCC/CPC/CIS
UBSS/CDC 27

(RUC : Record Update Counter)

(RUO: Record Update Originator)

data store copy

RUC RUO User record

12000 records max.
{

RRM : Retransmission Request Message
32 Ko max.
CAM
01/04/2
UIM : Update Initiation Messages
(CAM : Central Announcement Message) Version 6

MCC/CPC/CIS
UBSS/CDC 28


slave master slave
Appli- Appli-
Appli-
cation cation
cation

UIM

Appli- Appli-
cation cation
slave slave

01/04/2
Version 6

MCC/CPC/CIS
UBSS/CDC 29

CDC : periodic consistency checking

slave slave
master
Appli-
Appli- Appli-
cation
cation cation

CAM

Appli- Appli-
cation cation
slave slave

01/04/2
Version 6

MCC/CPC/CIS
UBSS/CDC 30

CDC : periodic consistency checking and recovery

At regular intervals (period is a CDC start-up parameter), the master initiates a consistency

check by broadcasting a special message.
The Central Announcement Message (CAM) contains the RUC and RUO fields of each

record of the Master datastore copy.
Each slave node performs the same algorithm.

- CDC requests retransmission from the master when :
RUC(CAM record) > RUC(datastore copy) (master has a new version)
RUC(CAM record) = RUC (datastore copy) and RUOs are different. (The
collision has not been resolved by application).
- CDC retransmits Update in broadcast mode when :
RUC(CAM record) < RUC(datastore copy) (master has an old version)
CDC retransmission requests are transmitted using a Retransmission Request Message

(RRM) with UDP point to point facility.
The master distributes its version of the record in question by mean of the Update

Retransmission Message (URM) with UDP broadcast facility.
01/04/2
Version 6

MCC/CPC/CIS
UBSS/CDC 31

CDC : retransmission request

slave slave
master
Appli-
Appli- Appli-
cation
cation cation

RRM

Appli- Appli-
cation cation
slave slave

01/04/2
Version 6

MCC/CPC/CIS
UBSS/CDC 32

CDC : Update Retransmission

slave slave
master
Appli-
Appli- Appli-
cation
cation cation

URM

Appli- Appli-
cation cation
slave slave

UIM : Update Initiation Messages 01/04/2
URM : Update Retransmission Message
Version 6

MCC/CPC/CIS
UBSS/CDC 33

CDC : update collision detection 1/2

The detection of the collision by CDC triggers the following actions


 first, the original contents of the local record is put in the collision store

 the received update of the record is placed in the datastore copy

 an update event is generated for all subscribed processes, except the originator of
the saved collision
 the originator of the saved collision which made the update, is notified that a
collision occurred.

It is the responsibility of the originator to recover from the collision


01/04/2
Version 6

MCC/CPC/CIS
UBSS/CDC 34

CDC : update collision detection 2/2

node A
node B
UIM
local update record R local update record R
RUC= n, RUO=A RUC= n, RUO=B

remote update record R remote update record R
RUC(UIM) = n, RUO(UIM) = B RUC(UIM) = n, RUO(UIM) = A

- node A and node B update the same record R in a close time
- CDC of node A(B) broadcasts an UIM message with record R update
- node A (B) receives an UIM message with a remote update of record R
- received record R has the same RUC and a different RUO as the local record R.
- a collision is detected
01/04/2
URM : Update Retransmission Message
Version 6

MCC/CPC/CIS
UBSS/CDC 35

CDC : behaviour in exceptional case

Failure of a slave has no influence for other slaves or the master.


Consistency of datastore copies is guaranteed if the master is available.


The master must have a high availability (located in a dual node).


When a master switches or is about to terminate it sends a Master Died Message

(MDM).
Absence of master is detected by reception of MDM or after no periodic reception of

CAM.
The event quot;master not alivequot; is then reported to the application.


The event quot;master alivequot; occurs when a slave detects a master (CAM reception).


01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 36

CDC : control node access

A user application is able to disconnect/reconnect the datastore copy from/to the LAN

using a library function
quot;Connectedquot; node access state is the normal CDC behavior. Local and remote (received)

updates are normally processed.
When the node access state is quot;isolatedquot;, received CDC messages are ignored. The local

update are only performed. UIM messages are not sent
After a transition from quot;isolatedquot; to quot;connectedquot;, the master records will overwrite local

records (regardless of the local RUC).
The quot;no_lan_supportquot; node access state is used when the CDC does not support LAN.

This state can not be changed

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 37

CDC : datastore design

Data normalization should be a solution for a good design


Three rules can be used:


1 : Remove repeating fields to a separate record
2 : Similarly, remove fields which can refer to another key
3 : Similarly, remove fields which can refer to other non-key
fields elsewhere
Try to avoid records which are bigger than the medium packet-size.


Don't forget that CDC does not support integrity neither between stores nor

between records of the same CDC store.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 38

CDC library functions 1/3

Datastore copy user registration:

cdc_open Open a specified datastore copy.
cdc_close Close a datastore copy.
cdc_subscribe Open a specified datastore copy and subscribe for CDC events.

Get and free exclusive datastore copy access:

cdc_store_acc Access to a datastore copy.

Create / Remove indexed sequential access environment:

cdc_is_build Create indexed sequential access environment.
cdc_is_remove Remove indexed sequential access environment.

Controlling master/slave state:


cdc_master_slave Change status of datastore copy.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 39


Accessing datastore copy records:


cdc_record_acc() Get physical access to a datastore copy record.
cdc_coll_acc() Get physical access to a saved collision record.
cdc_direct_read() Obtain a copy of a datastore copy record by record number.
cdc_direct_coll_read() Obtain a copy of a saved collision record by record number.
cdc_direct_update() Update a datastore copy record by record number.
cdc_direct_delete() Delete a datastore copy record by record number.
cdc_seq_read() Obtain datastore copy records sequentially by record number.
cdc_is_read() Obtain a copy of a datastore copy record by key.
cdc_is_update() Update a datastore copy record by record number.
cdc_is_create() Create a ’new’ record using a deleted one.
cdc_is_key_delete() Delete all datastore records with a specified key.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 40


Handling events:

cdc_event Get next event from the event chain.
cdc_upd_recnum Get number of next updated record.
cdc_coll_recnum Get number of next save collision record.
cdc_event_ctl Control sequence of event reporting.

Retrieving datastore copy information:

cdc_nr_records Get number of records in datastore copy.
cdc_size_record Get the size of the datastore copy records.
cdc_statistics Get/reset statistics.
cdc_store_acc_status Get the store access status.
cdc_req_cons_rep Request consistency reply message.
cdc_is_key_exists Check for existence of datastore records with a specified key.

Controlling node access state:

cdc_control_node_access Change status of node access.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 41

CDC event management
CDC event notiification

FIFO
Application
CDC Process
Application
FIFO
Process Process
Shared memory

CDC event
CDC events chain
RUC RUO User record

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 42

CDC Reader Side Typical use
declaration of FIFOCDC with their associated type
initialization of QUEUE_DECK
initialization of FIFOCDC
declaration of mask
initialization of CDC utilisation
OPEN FIFOCDC for system wide communication
initialization of mask with priority of FIFOCDC=1
OPEN CDC
recover events
do forever
WAIT_FROM(mask)
case receiving FIFO id of
FIFOCDC :
do forver
case CDC event of
NOEVENT : exit
UPDATE : treament 2
COLLISION : treatment 3
.....
end case
end do
time out : treatment
end case
end do
01/04/2
RRM : Retransmission Request Message Version 6

MCC/CPC/CIS
UBSS/CDC 43

CDC debugger : CDD 1/2

CDD is a tool to inspect and /or change contents or behavior of one or more CDC stores


CDD is a kind of application program and must be specified in the (ncf.n) Node

Configuration File. It can be actived by an interactive session.
CDD main commands :

close / open close / open a CDC store.
define format define one or all records from a CDC store.
delete delete one or all records from a CDC store.
force close / unlock force one or all processes to close or unlock a CDC store.
format assign a format to a CDC store.
get events display events that are currently received for a CDC store.
isolate / reconnect disconnect / reconnect the CDC store from lan.
lock / unlock lock / unlock a CDC store.
log events enable or disable the logging of events.
master / slave become master / slave of a CDC store.
read / write read from or write to a CDC store.

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 44

CDC debugger : CDD 2/2

CDD main commands (continued):

recover events force CDC to send events for all modified records of a CDC store.
reset statistics reset the statistical information, kept by CDC for a CDC store.
show users / status show the users or status and locks of one or all CDC stores.
skip events skip all events that are currently received for a CDC store.
use use a CDC store to be the current active CDC store .
wait consistency wait for (a record of) the CDC store to become consistent.
wait events wait for and display the receipt of events for a CDC store.

CDD offers others general commands


log input to a logfile
log output and error information to a logfile
provides options for starting with a commandfile
...

01/04/2

Version 6

MCC/CPC/CIS
UBSS/CDC 45

Cdc

Recommended

Recommended

More Related Content

More from Emmanuel Fuchs

More from Emmanuel Fuchs (20)

Recently uploaded

Recently uploaded (20)

Cdc