The document discusses a novel approach to distributed multimedia synchronization using fuzzy causal relations, proposing that strict causal order may not be necessary for applications allowing some degradation. It outlines a fuzzy causal relation model that improves synchronization performance by establishing a degree of cause-effect between events based on spatial, temporal, and logical distances. The proposed model can operate asynchronously and provide qualitative performance measures without halting the system.

1. Congreso Internacional de Informática Aplicada
Distributed Multimedia Synchronization
Based on Fuzzy Causal Relations
Luis Alberto Morales Rosales
Misantla, Veracruz, a 25 de Abril del 2012

2. Overview
• Practical Aspect (application area)
– Distributed multimedia synchronization in real time
• Distributed sources
• Heterogeneous data (discreet and continuous)
• Loss of data
• Transmission delay
• Theoretical Aspect
– Development of a flexible causal relation for distributed systems
unlike the one proposed by Lamport (1978)
• For applications where certain degradation of the system is allowed
is not necessary to assure a strict causal delivery, for example,
multimedia, scheduling, cooperative work and planning
2

3. Outline
1. Introduction
2. Related work
3. Problem description
4. Research proposal
5. Results
6. Conclusions
INAOE 3

4. Introduction
• Distributed systems
– Absence of a global clock
– Concurrency
– Fail tolerance
– Examples: cooperative applications, mobile systems,
multimedia applications, etc.
• Distributed multimedia systems
– The exchange of big volumes of multimedia data in a
communication network among a group of participants
4

5. Introduction
• Multimedia synchronization
– The preservation of temporal dependencies
among the application data from the time of
generation to the time of presentation
Audio Video
Multimedia
Text, slides, images, etc.
Multimedia data
5

6. Introduction
A
V
V
P1 P2
A
A V S S
A V
V
V
A- Audio (Voice) P1 – Participant 1
V- Video P2 - Participant 2
S- Slides P3 P3 - Participant 3
Example of a multimedia scenario
6

7. Introduction
A
P1
V
V
P2
A
V
P3
S
A- Audio (Voice) P1 – Participant 1
V- Video Time Line
P2 - Participant 2
S- Slides P3 - Participant 3
Representation of a multimedia scenario
7

8. Related Work
Synchronization
Synchronous Asynchronous
Common reference Logical dependencies
Physical clock
Disadvantages
Disadvantages Introduction of random delays
Bottlenecks Discard of messages
Introduction of random delays
Halt of the system
Not scalable
8

9. Related work: Synchronous
• Works that use fuzzy logic
– Zhou et al. [6]
• Temporal model based on fuzzy petri networks to
represent the multimedia synchronization
• Video on demand
– Ramaprabhu et al. [3]
• Broadcast transmission of video on demand
• Consider the parameters: available bandwidth, network
delay and buffer space
9

10. Related work: Synchronous (Cont.)
• Works that use fuzzy logic
– Coelho et al. [1]
• Methodology for the high level specification and
decentralized coordination of temporal interdependences
among objects of multimedia documents previously
stored
• Membership function to calculate the lifetime of the data
• Strict causal relation for event ordering
• Global time reference
10

11. Related work: Asynchronous
Causal and ∆-causal algorithms
• The causal and ∆-causal algorithm use the causal relation
proposed by Lamport
– Main works: Baldoni [2] et al., Tomoya et al.[5] and Pomares et al. [4]
– Introduce delays and/or discard of messages (packets)
a b c
Time Line
Strict delivery order ∆-causal order
b→c b→c and ∆c < ∆max
( b, c) ( b, c )
Infinite halt Deliver c and discard b
11

12. Context problem
• Distributed multimedia synchronization in real
time by considering:
• Asynchronous sources
• Heterogeneous data
• Communication network characteristics
– Loss of the data and transmission delay
• Without previous knowledge of the system
behavior
How can we ensure the temporal dependencies
among events in this kind of enviroments?
12

13. Research proposal
• As hypothesis we claim that for systems where some
degradation of the system is allowed, it is not
necessary ensuring a strict causal order of the data,
which is associated with a binary value
• We propose the fuzzy causal relation and the fuzzy causal
consistency to relax the order delivery
• We show that relaxing the causal delivery order of the data
can improve the performance of the synchronization
13

14. Results
Fuzzy causal relation (FCR)
• Fuzzy causal relation (FCR) denoted by “a λ b”
→
– The FCR is based on a notion of “distance” among the events
– The distance can be established considering three main domains: spatial
(RS), temporal (RT) and/or logical(RL)
– Using the notion of distance, the FCR establishes a cause-effect
degree that indicates “how long ago” an event a happened before
an event b
– One assumption considered for the FCR is that “closer” events
have a stronger cause-effect relation, according to the addressed
problem
14

15. Results (Cont.)
Fuzzy causal relation (FCR)
• The distance between events is determined by the fuzzy
relation DR: E × E → [0, 1]
DR(a,b) = RS ∪ RT ∪ RL
Where:
RS=(R1 ∪ R2∪... ∪ Rs)
RT=(R1 ∪ R2∪... ∪ Rt)
RL=(R1 ∪ R2∪... ∪ Rl)
Ri = membership function
• The fuzzy union operator chosen for intra and inter
domains is the max operator
15

16. Results (Cont.)
Fuzzy causal relation (FCR)
• Distance relation
– The DR grows monotonically and it is directly proportional to the
spatial, temporal and/or logical distances between a pair of
events
– DR(a,b) with a value tending to zero indicates that the events a
and b are “closer”
– The DR cannot determine precedence dependencies among
events, it only indicates certain distance among them
16

17. Results (Cont.)
Fuzzy causal relation (FCR)
• The fuzzy causal relation “ λ on a set of events E satisfies
→”
the two following conditions:
3. a λ b If a→ b ∧ 0 ≤ DR(a,b) < 1
→
FCR between two events
λ
6. a → c If ∃b  a→b→c ∧ DR(a,b) ≤ DR(a,c) : DR(a,b), DR(a,c) < 1
Transitivity
•The fuzzy concurrency (FCNR) is defined as follows:
The value of the fuzzy causal relation
a λbetween a pair of events is represented as: FCR(a,b)
b If ¬ (a→b ∨ b→a) ∧( (DR(a, b)= DR(b, a) )< 1)
17

18. Results(Cont.)
Example of the FCR and the FCNR
2
FCR(a,e)
b e
1
a c f
d
Example of fuzzy precedence among causal messages FCR(a,e)
λ
a → e If ∃b  a→b→e ∧ DR(a,b) ≤ DR(a,e) : DR(a,b), DR(a,e) ) < 1
18

19. Results(Cont.)
Fuzzy Causal Consistency
• The Fuzzy Causal Consistency (FCC) is based on the
FCR
• The goal of the FCC is to indicate “how good” the
performance of the system is in a certain time
• The meaning of the performance can be indicated
according to the problem to solve
19

20. Results(Cont.)
Fuzzy Causal Consistency
FCRp (b,a)
b
c
d
FCRp(c,a) ∑ GP(b) FCR (b, a) p Fuzzy causal
H(a) FCRp(d,a) b∈H ( a )
FCC p (a) = consistency
FCRp( w,a) ∑ GP(b) FCCp(a)
w b∈H ( a )
Calculation of the Fuzzy causal consistency
b
GP(b) is a weighting degree used to determine priorities or weight
FCRp( b,d)
a for every fuzzy causal relation when it is needed
d Average
FCRp( c,d) FCCp(a)
Weight
c
Example of calculation of the FCC
20

21. Results (Cont.)
Fuzzy causal delivery order
• The fuzzy causal delivery order must satisfy the following
condition:
λ
If send(m) → send(m’) then
∀p ∈ dests(m)∩dests(m’), FCCp(m’)≤ FCCmax
– deliveryp(m) → deliveryp(m’) or
– deliveryp(m’) → deliveryp(m)
21

22. Results
Causal order vs fuzzy causal order
a+ c+ = xn a -→a +
i =Part(X) A C
a- = x1 c- a+→ d-
d = y1 -
B b =yp
+
j =Part(Y) D Causal delivery
order of events
a+
d -
Fuzzy causal delivery
k =Part(Z)
Synchronization Period
Example of a fuzzy causal delivery
Fuzzy causal order
Strict order delivery ∆-causal order delivery delivery
a+→d - a+→d - y ∆d < ∆max + λ
a →d -
( a+, d - ) ( a+, d - ) Deliver: (a+, d - ) or
Infinite halt Deliver d and discard a
(d -, a+)
22

23. Synchronization Scheme of the Distributed Multimedia Mechanism
Model
Fuzzy Causal Component
Fuzzy Causal
Temporal Fuzzy Causal Relations Consistency
Distance FCRp(m) ∀m ∈ H(a) FCCp(a)
RN
Performance of
the System
Causal Distance
RD
Fuzzy Control System
Weighting
Degree
Fuzzy
GP
Control
Network Adjustment and
Conditions Selective Discard
System
NC
Component of Current State of
input variables the System
23

24. Synchronization model
A E C E’
i =Part(X)
j =Part(Y) F D F’ B
i≠j
Synchronization periods
Fuzzy causal delivery
A → I EF → I (C ||| D) → I E ' F ' → I B
Formally a period is defined as: EF λ f ∨ eλ
⇒ ∀(e, f) ∈ (E x F), e → f
24

25. Synchronization model
25

26. Results (Cont.)
The FCR applied to the intermedia synchronization
• The FCR for the intermedia synchronization problem gives
a qualitative measure of the temporal and logical
dependencies between two events with regard to a partial
view of a participant
• The value of the FCR is calculated by:
λ
FCR (a, b) = DR (a, b) if a  →b

DR (a, b) = max( RD , RN )
26

27. Results (Cont.)
The FCR applied to the intermedia synchronization
• The meaning of the values obtained by the
FCR(a, b) is as follows:
– When FCR(a, b) tends to zero indicates that the events
a and b are “closer”, which means that the network
present low transmission delay and low loss of
messages
– When FCR(a, b) tends to the unit indicates the opposite
27

28. Results (Cont.)
FCC applied to the intermedia synchronization
• FCC gives a qualitative measure of the synchronization
error, according to the temporal and logical dependencies
in the whole system in a certain time, with regard to the
partial view that every participant has
– When FCCp(a) tends to zero, this indicates that the
performance of the system is good
– When FCCp (a) tends to the unit, this indicates that the
system performance is regular or bad
28

29. Results (Cont.)
Fuzzy control system
Fuzzy Causal
Current State
Consistency, FCCp(a) µ= delivery time
Fuzzy of the System
Control System
Network Conditions (CS)
NC(a)
Diagram of the fuzzy control system
• The NC determines the network condition
• TheThe NC consider:control will bedelays, jitter among messages, loss of
– actions that the transsmision able to execute are:
– the immediate delivercongestion and message aavailable
messages, network of the received bandwith or its delay, and
– the determination whether a selective discard of the messages contained
– in the causal history of the messageto is carried out
When the value of NC(a) is near a zero, this means that the network
• Theconditions are good is used as inference mechanism
Mandami’s model
• – When membership one, this are included the network conditions
TriangularNC(a) tends to functions represents that as part of the fuzzy
control bad
are
29

30. Results (Cont.)
Test of a multimedia scenario
• The Host from W to Y
send:
– Audio
– Video, and
– Animation Host
Z
• Host Z only listens
• We simulated three
scenarios:
• Soft, Medium and Hard Host Host Host
W X Y
• 100,000 simulations
• Fuzzy causal order vs.
∆-causal order
•When Lamport’s relation is used the system halts if a message is lost
30

31. Results (Cont.)
Test: Fuzzy causal order vs. ∆-causal
Soft case
Big without discard
Medium without discard
Classification of delivered messages
31

32. Results (Cont.)
Test: Fuzzy causal order vs. ∆-causal
Soft case
∆-Causal Order
Fuzzy causal
order
Classification of messages loss
INAOE 32

33. Results (Cont.)
Test: Fuzzy causal order vs. ∆-causal
Medium case
Medium without
Small with discard
discard
Classification of delivered messages
33

34. Results (Cont.)
Test: Fuzzy causal order vs. ∆-causal
Medium case
∆-Causal Order
Fuzzy causal
order
Classification of messages loss
34

35. Results (Cont.)
Test: Fuzzy causal order vs. ∆-causal
Hard case
∆-Causal Order
Fuzzy causal
order
Classification of messages loss
35

36. Conclusions
• The definitions of FCR and FCC permits to establish a
more asynchronous ordering for application where certain
degradation of the system is allowed
• The FCO allows a more asynchronous delivery of events
compared with the causal delivery order based on the
happened-before relation introduced by Lamport
• The FCR and FCC allow a measure of the performance of
the application by a participant at runtime without halt the
system
36

37. Conclusions
• A novel fuzzy control for intermedia
synchronization that works in a distributed
manner was presented
• By using the fuzzy control and FCC the
mechanism discard fewer messages than the
∆-causal mechanism

38. References
1. André L. V. Coelho, Alberto B. Raposo, Ivan L. M. Ricarte. “Bringing Flexibility to
the Specification and Coordination of Temporal Dependencias among Multimedia
Components”. VII Simposio Brasileiro de Sistemas Multimídia e Hipermídia,
Florianópolis, Brazil, SBC. 2001, pp. 37-52.
2. Roberto Baldoni, Achour Mostéfaoui, Michel Raynal. “Efficient Causally Ordered
Communications for Multimedia Real-Time Applications”. The 4th International
Symposium on High Performance Distributed Computing (HPDC '95),
Washington, DC, USA, August 2-4, 1995, pp. 140-147
3. Ramaprabhu Janakiraman, Marcel Waldvogel and Lihao Xu. “Fuzzycast: Efficient
Video-on-demand over Multicast”. Proceedings INFOCOM 2002, New York, NY,
USA, June 2002.
4. Saul E. Pomares Hernandez, Luis A. Morales Rosales, Jorge Estudillo Ramirez,
and Gustavo Rodriguez Gomez, “Logical Mapping: An Intermedia Synchronization
Model for Multimedia Distributed Systems,” Journal of Multimedia, Eds. Academy
Publisher, Vol. 3 No.5, 2008, ISSN: 1796-2048, pp. 33-41.
5. Tomoya Enokido, Sei-ichi Hatori, Takuya Tojo Makoto Takizawa. “Group
Communication in Distributed Multimedia Objects”. Proceeding of The Eighth
IEEE International Workshop on Object-Oriented Real-Time Dependable Systems
(WORDS 2003), 2003, pp. 258.
6. Yi Zhou, Tadeo Murata. “Modeling and Analysis of Distributed Multimedia
Synchronization by Extended Fuzzy-Timing Petri Nets”. Transactions of the
Society for Desing and Process Science, Volume 5, Number 4, December 2001,
pp. 23-37.
38

39. Thanks !
Questions?
39

40. Resultados
1
Causalidad estricta
Relación causal difusa y
consistencia causal difusa
0 Línea del tiempo
Restricciones de una causalidad estricta vs.
relación causal difusa y consistencia causal difusa
INAOE 40

41. Related work
Characteristics Coelho et. al Proposal
Predefined synchronization points Yes No
Decisions Centralized Distributed
Real time No Yes
Communications Producer-Consumer Group
Time Constraints No Yes
Quality of services No Yes
Clocks to order the events Physics Logics
Comparison of characteristics between the work done by
Coelho et al. in [3] and our proposed research
INAOE 41

42. Algoritmo Periodo 1
A a’
Part(w), C Historia Causal H(m) de a’:
a {d,f,z}
D d B E
Part(x)
Periodo 2
f G
Part(y) F H
g
z
Part(z)
6.- Calculo del estado actual
Part(y) orden de entrega causal:
1.- Calculo de los Periodos de cada elemento contenido en H(m) 7.- Calculo de CR
2.- Obtención de los parámetros RTT, Bw, jitter p/c evento de H(m) 8.- Paso de parámetros
3.- Calculo de la Distancia Temporal: t(a’)-t(a) al control difuso (CR y CCD)
9.-Salida del control difuso
4.- Calculo de la Distancia Causal (a’): {d,f,z} •Retrasar
5.- Calculo de RCD(a’) y CCD a partir de H(a’)={d,f,z} •Entregar
INAOE •Descarte Selectivo 42

43. • The GP variable is the weighting grade that determines the
degradation of a channel (participant) according to the best
channel of the system
 ( RTTi + jitteri ) + PPi 
GPi =  
 BChannel 
BChannel = min in=1 [ ( RTTi + jitteri ) + PPi ]

44. 95 190 250 Global Time
0
mj-1 mi H(mi)={mj, mk}
i =Part(A) audio A
mj
B
j =Part(B) video
k =Part(C) audio C
mk-2 mk-1 mk
Δ(mk-2) Δ(mj-1)
m1
l

45. Media Mode, application Maximum Error
synchronization
(QoS, dmax)
Video Animation Correlated ±120ms
Audio Lip synchronization ±80ms
Image Overlay ±240ms
No-overlay ±500ms
Text Overlay ±240ms
No-overlay ±500ms
Audio Animation Event correlation (e.g. dancing) ±80ms
Audio Tightly coupled(stereo) ±11ms
Loosely coupled (dialogue mode with ±120ms
various participants)
Loosely coupled (e.g. background ±500ms
Image music) coupled (e.g. music with notes)
Tightly ±5ms
Loosely coupled (e.g. slide show) ±500ms
Text Notes of text ±240ms
Pointer Audio related to the item ±700ms
Tabla 2. Maximum error tolerable for multimedia synchronization
INAOE 45

46. Algorithm
A a’
Part(w), C Historia Causal H(m) de a’:
a {d,f,z}
D d B E
Part(x)
f G
Part(y) F H
g
z
Part(z)
Part(y) orden de entrega causal: 6.- Calculo del estado
actual
7.- Calculo de CR
1.- Calculo de los Periodos de cada elemento contenido
en H(m) 8.- Paso de parámetros
2.- Obtención de los parámetros RTT, Bw, jitter p/c evento de H(m)
3.- Calculo de la Distancia Temporal: t(a’)- al control difuso (CR y CCD)
t(a)Calculo de la Distancia Causal (a’): 9.-Salida del control difuso
4.- •Retrasar
{d,f,z}
5.- Calculo de RCD(a’) y CCD a partir de •Entregar
H(a’)={d,f,z} INAOE •Descarte Selectivo
46

47. 95 190 250 Global Time
0
mj-1 mi H(mi)={mj, mk}
i =Part(A) audio A
mj
B
j =Part(B) video
k =Part(C) audio C
mk-2 mk-1 mk
Δ(mk-2) Δ(mj-1)
m1
l

48. Example of a synchronization
multimedia scenario
95 165 253 Global Time
0
mj-1 mi H(mi)={mj, mk}
i =Part(A) audio A
mj
B
j =Part(B) video
k =Part(C) animated C
mk-2 mk-1 mk
images
Δ(mk-2) Δ(mj-1)
m1
l

49. Results(Cont.)
Fuzzy causal relation
1 1
Cause-effect Cause-effect
Temporal cause-effect Logical cause-effect degree
degree
INAOE 49

50. Related work
Characteristics Coelho et. al Proposal
Predefined synchronization points Yes No
Decisions Centralized Distributed
Real time No Yes
Communications Producer- Group
Consumer
Time Constraints No Yes
Quality of services No Yes
Clocks to order the events Physic Logic
Comparison of characteristics between the work done by
Coelho et al. in [3] and our proposed research
INAOE 50

51. Results(Cont.)
The FCR applied to the intermedia synchronization
• The membership functions, “RN” and “RD”
can be calculated by a triangular function
1
defined as:
0 If x< f
x− f
A(x) If f≤ x< hO f h
h− f
1 If x≥h
INAOE 51

52. Results(Cont.)
The FCR applied to the intermedia synchronization
1 1
Cause-effect Cause-effect
dmax Causal distance
Temporal cause-effect degree Logical cause-effect degree
dmax is assigned according to the maximum error allowed
Causal distance is established to four events
INAOE 52

53. Results (Cont.)
Input value for the fuzzy causal relation
• Temporal distance
m m’
Part(X)
∆(m)
Part(Y)
Local physical clock of Part(Y)
∆(m)= time_arrival(m) – time_arrival(m’) ∀p∈ H(m) ∧ m = last_messagep
INAOE 53

54. Results(Cont.)
FCC applied to the intermedia synchronization
 GP is the weighting grade that determines the degradation
of a channel (participant) according to the best channel of
the system
 ( RTTi + jitteri ) + PPi 
• RTT is the Round trip time of a
message
GPi =  
 BChannel 
• Jitter is the fluctuation of end to
end of a message with the next
message inside the same stream
BChannel = min in=1 [ ( RTTi + jitteri ) + PPi ]
• PP is the number of lost
messages in a synchronization
period
• n is the number of channels
(participants) which are causally
related to the event a
INAOE 54

55. Results (Cont.)
Input value for the fuzzy control system
• NC is calculated by the following formula:
n  2 Bi  ( PEi − PRi ) 
∑  RTT

i =1 
+
 PEi
 * GPi
i  
NC (a ) =
S GP
• Where
– B is the bandwidth available in the network.
– RTT is the round trip time of a message a across the network.
– PE is the number of expected messages inside the synchronization period.
– PR is the number of received messages inside the synchronization period.
– SGP is the sum of the weighted grades of the channels.
– n is number of channels(participants) which are causally related with the message
a, known as the causal history of a.
INAOE 55

56. Results (Cont.)
Fuzzy control system
• The set of rules used by the fuzzy control to adjust
the delivery time of the message a has the form:
If NC(a) ∈ Ai y FCCp(a) ∈ Bi then µ ∈Ci
• Where:
– NC(a), FCCp(a) and µ are the enter parameters of the fuzzy
control system
– Ai = {good, regular, bad} are the linguistic labels for NC(a)
– Bi = {good, regular, bad} are the linguistic labels for the FCCp(a)
– Ci = {big, medium, short} are the linguistic labels for the output µ
INAOE 56

57. Results (Cont.)
Fuzzy control system
• The set of rules that the fuzzy control system considers is the
following
3. If NC(a)= bad and FCCp(a)= good, then µ= medium (delivery without discard)
4. If NC(a)= bad and FCCp(a) = regular, then µ= short (delivery with discard)
5. If NC(a)= bad and FCCp(a) = bad, then µ= short (delivery with discard)
6. If NC(a)= regular and FCCp(a) = good, then µ= medium (delivery without discard)
7. If NC(a)= regular and FCCp(a) = regular, then µ= medium (delivery without discard)
8. If NC(a)= regular and FCCp(a) = bad, then µ= short (delivery with discard)
9. If NC(a)= good and FCCp(a) = good, then µ= big (delivery without discard)
10. If NC(a)= good and FCCp(a) = regular, then µ= big (delivery without discard)
11. If NC(a)= good and FCCp(a) = bad, then µ= medium (delivery without discard)
• The Mamdani model has been selected as the inference
mechanism
• The centroide function has been selected for the defuzzyfication of
the output of the fuzzy control system
INAOE 57