Adaptive Internal Clock Synchronization

1,593 views

Published on

Existing clock synchronization algorithms assume a
bounded clock reading error. This, in turn, results in an inflexible design
that typically requires node crashes whenever the given bound might be
violated. We propose a novel, adaptive internal clock synchronization
algorithm which allows to compute the deviation between the clocks during
runtime. The computed deviation can be propagated to the application layer
to allow it to adapt its behavior according to the current clock deviation.
The contributions of this paper are: (1) a new specification of a relaxed
clock synchronization problem, and (2) a new clock synchronization algorithm
with a novel approach to dealing with crash failures.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,593
On SlideShare
0
From Embeds
0
Number of Embeds
52
Actions
Shares
0
Downloads
24
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Adaptive Internal Clock Synchronization

  1. 1. Motivation Background Adaptive Clock Sync. Evaluation Summary 1 of 25 slides Adaptive Internal Clock Synchronization SRDS 2008 Zbigniew Jerzak, Robert Fach, and Christof Fetzer October 6, 2008 {Zbigniew.Jerzak, Robert.Fach, Christof.Fetzer}@inf.tu-dresden.de Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  2. 2. Motivation Background Adaptive Clock Sync. Evaluation Summary 2 of 25 slides Introduction Build safer distributed systems using weaker assumptions [FC03] In this paper/presentation: Relaxed specification for internal clock synchronization New clock synchronization algorithm New way to deal with crash failures Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  3. 3. Motivation Background Adaptive Clock Sync. Evaluation Summary 3 of 25 slides Current Approach 1. Assume a bounded deviation ∆max 2. However, in timed asynchronous applications: messages get dropped or are arbitrarily delayed different nominal and observed oscillator frequencies clocks’ speeds fluctuate with time processes crash 3. ...thus 1 cannot be guaranteed Result: An inherently inflexible and a failure-prone system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  4. 4. Motivation Background Adaptive Clock Sync. Evaluation Summary 3 of 25 slides Current Approach 1. Assume a bounded deviation ∆max 2. However, in timed asynchronous applications: messages get dropped or are arbitrarily delayed different nominal and observed oscillator frequencies clocks’ speeds fluctuate with time processes crash 3. ...thus 1 cannot be guaranteed Result: An inherently inflexible and a failure-prone system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  5. 5. Motivation Background Adaptive Clock Sync. Evaluation Summary 3 of 25 slides Current Approach 1. Assume a bounded deviation ∆max 2. However, in timed asynchronous applications: messages get dropped or are arbitrarily delayed different nominal and observed oscillator frequencies clocks’ speeds fluctuate with time processes crash 3. ...thus 1 cannot be guaranteed Result: An inherently inflexible and a failure-prone system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  6. 6. Motivation Background Adaptive Clock Sync. Evaluation Summary 3 of 25 slides Current Approach 1. Assume a bounded deviation ∆max 2. However, in timed asynchronous applications: messages get dropped or are arbitrarily delayed different nominal and observed oscillator frequencies clocks’ speeds fluctuate with time processes crash 3. ...thus 1 cannot be guaranteed Result: An inherently inflexible and a failure-prone system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  7. 7. Motivation Background Adaptive Clock Sync. Evaluation Summary 4 of 25 slides Our Approach No upper bound for the deviation (∆max ) compute deviation between processes in real-time use computed deviation to adapt system behavior No bound on the maximum number of crashed processes mask crashes using clock readings extrapolation Result: An adaptive, safer system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  8. 8. Motivation Background Adaptive Clock Sync. Evaluation Summary 4 of 25 slides Our Approach No upper bound for the deviation (∆max ) compute deviation between processes in real-time use computed deviation to adapt system behavior No bound on the maximum number of crashed processes mask crashes using clock readings extrapolation Result: An adaptive, safer system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  9. 9. Motivation Background Adaptive Clock Sync. Evaluation Summary 4 of 25 slides Our Approach No upper bound for the deviation (∆max ) compute deviation between processes in real-time use computed deviation to adapt system behavior No bound on the maximum number of crashed processes mask crashes using clock readings extrapolation Result: An adaptive, safer system Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  10. 10. Motivation Background Adaptive Clock Sync. Evaluation Summary 5 of 25 slides Use Case p is assigned a time slot [S, T ] for access to R [S, T ] defined w.r.t. some abstract clock p is not synchronized with a priori known ∆max , instead... Use Ep – maximum local clock synchronization error an upper bound on the deviation from some abstract clock calculated by every process propagated to application layer for error handling p uses R only within [S + Ep , T − Ep ] p knows how well it is synchronized p knows when it is allowed to access R Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  11. 11. Motivation Background Adaptive Clock Sync. Evaluation Summary 5 of 25 slides Use Case p is assigned a time slot [S, T ] for access to R [S, T ] defined w.r.t. some abstract clock p is not synchronized with a priori known ∆max , instead... Use Ep – maximum local clock synchronization error an upper bound on the deviation from some abstract clock calculated by every process propagated to application layer for error handling p uses R only within [S + Ep , T − Ep ] p knows how well it is synchronized p knows when it is allowed to access R Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  12. 12. Motivation Background Adaptive Clock Sync. Evaluation Summary 5 of 25 slides Use Case p is assigned a time slot [S, T ] for access to R [S, T ] defined w.r.t. some abstract clock p is not synchronized with a priori known ∆max , instead... Use Ep – maximum local clock synchronization error an upper bound on the deviation from some abstract clock calculated by every process propagated to application layer for error handling p uses R only within [S + Ep , T − Ep ] p knows how well it is synchronized p knows when it is allowed to access R Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  13. 13. Motivation Background Adaptive Clock Sync. Evaluation Summary 6 of 25 slides Processes and Hardware Clocks based on TADSM [CF99] Set of N = {p0 , p1 , . . . , pn } processes Connected via communication bus Hardware clocks – Hp (t): bounded drift: ∀p∈N ∀t : |ρp (t)| ≤ ρmax within linear envelope of real time: (t − s)(1 − ρmax ) ≤ Hp (t) − Hp (s) ≤ (t − s)(1 + ρmax ) Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  14. 14. Motivation Background Adaptive Clock Sync. Evaluation Summary 8 of 25 slides Generic Internal Clock Synchronization Algorithm 1 C l o c k V a l Ap ; // c u r r e n t a d j u s t m e n t 2 C l o c k V a l T ; // end o f c u r r e n t r o u n d 4 void i n i t () { 5 ( Ap ,T) = i n i t i a l A d j u s t m e n t ( ) ; 6 // e v e r y P s t a r t i n g a t T 7 schedule ( synchronizer , P, T) ; 8 } 10 void synchronizer () { 11 // N − number o f p r o c e s s e s 12 ClockVal c l k [N] , 13 ClockVal e r r [N ] ; 14 // r e m o t e c l o c k r e a d i n g 15 readClocks ( clk , e r r ) ; 16 // a d j u s t m e n t f o r t h e n e x t r o u n d 17 Ap = a d j u s t ( Ap , T , c l k , e r r ) ; 18 // s e t T t o n e x t r o u n d 19 T = T + P; 20 } Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  15. 15. Motivation Background Adaptive Clock Sync. Evaluation Summary 9 of 25 slides Software Clocks and Remote Clock Reading We do not apply Ap directly to the Hp (t) Instead we use software clocks: Sp (t) = Hp (t) + ap (t) Ap is calculated based on the remote clock readings probabilistic remote clock reading [FC99b] provides values of the remote clocks clk[] and accompanying remote clock reading errors err[] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  16. 16. Motivation Background Adaptive Clock Sync. Evaluation Summary 10 of 25 slides Failure Model Messages are supposed to be delivered within δ might be delayed delivered out of order get dropped Arbitrary hardware clock failures: |ρp (t)| > ρmax converted to stop failures [FC99a] Processes’ value (byzantine) failures handled by Software Encoded Processing [WF07] Specifically: No upper bound on the frequency of communication and process failures Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  17. 17. Motivation Background Adaptive Clock Sync. Evaluation Summary 10 of 25 slides Failure Model Messages are supposed to be delivered within δ might be delayed delivered out of order get dropped Arbitrary hardware clock failures: |ρp (t)| > ρmax converted to stop failures [FC99a] Processes’ value (byzantine) failures handled by Software Encoded Processing [WF07] Specifically: No upper bound on the frequency of communication and process failures Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  18. 18. Motivation Background Adaptive Clock Sync. Evaluation Summary 11 of 25 slides Problem Statement ∀p,q∈N : Sp (t) = Sq (t) (1) However, achieving 1 is not possible Minimize & bound difference between software clocks: ∀p,q∈N:p,q ¬crashed(t) ∀t : |Sp (t) − Sq (t)| ≤ Ep (t) + Eq (t) Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  19. 19. Motivation Background Adaptive Clock Sync. Evaluation Summary 11 of 25 slides Problem Statement ∀p,q∈N : Sp (t) = Sq (t) (1) However, achieving 1 is not possible Minimize & bound difference between software clocks: ∀p,q∈N:p,q ¬crashed(t) ∀t : |Sp (t) − Sq (t)| ≤ Ep (t) + Eq (t) Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  20. 20. Motivation Background Adaptive Clock Sync. Evaluation Summary 11 of 25 slides Problem Statement ∀p,q∈N : Sp (t) = Sq (t) (1) However, achieving 1 is not possible Minimize & bound difference between software clocks: ∀p,q∈N:p,q ¬crashed(t) ∀t : |Sp (t) − Sq (t)| ≤ Ep (t) + Eq (t) q Abstract Clock p Eq Ep Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  21. 21. Motivation Background Adaptive Clock Sync. Evaluation Summary 12 of 25 slides Ep – Maximum Clock Synchronization Error Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) cfn() = mid Lip , Up i determines the Ap ← − → − Ep (T ) = max cfn() − M p (T ), M p (T ) − cfn() determines the midpoint estimation error Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  22. 22. Motivation Background Adaptive Clock Sync. Evaluation Summary 12 of 25 slides Ep – Maximum Clock Synchronization Error Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) cfn() = mid Lip , Up i determines the Ap ← − → − Ep (T ) = max cfn() − M p (T ), M p (T ) − cfn() determines the midpoint estimation error Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  23. 23. Motivation Background Adaptive Clock Sync. Evaluation Summary 13 of 25 slides Convergence Functions: cfn() = mid Lip , Up i Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) A convergence function: determines an abstract clock value which... ...should be reached at the end of the next round Specifically: cfnV 0 () — based on [WL88] cfnV 1 () — based on [CF94] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  24. 24. Motivation Background Adaptive Clock Sync. Evaluation Summary 13 of 25 slides Convergence Functions: cfn() = mid Lip , Up i Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) A convergence function: determines an abstract clock value which... ...should be reached at the end of the next round Specifically: cfnV 0 () — based on [WL88] cfnV 1 () — based on [CF94] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  25. 25. Motivation Background Adaptive Clock Sync. Evaluation Summary 14 of 25 slides Convergence Functions: cfn() = mid Lip , Up i Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) Perfect World (no transmission delays, no drift): p Cq(T,p) Cr(T,p) cfn() Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  26. 26. Motivation Background Adaptive Clock Sync. Evaluation Summary 14 of 25 slides Convergence Functions: cfn() = mid Lip , Up i Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) Real Life (transmission delays, drift): p Cq(T,p) Cr(T,p) εq(T,p) εr(T,p) cfn() Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  27. 27. Motivation Background Adaptive Clock Sync. Evaluation Summary 15 of 25 slides → − ← − Midpoint Estimation Error: M p (T ), M p (T ) Ep (T ) = |Cp (T , p) − cfn()| + Ep (T ) Possible values a midpoint can take: ← − ← − ← − M p (T ) = mid L p (T ), U p (T ) → − − → − → M p (T ) = mid L p (T ), U p (T ) cfn() ← → Mp() Mp() Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  28. 28. Motivation Background Adaptive Clock Sync. Evaluation Summary 16 of 25 slides Extrapolation & Intersection Re-use clock readings from previous round Mask transient clock reading failures (extrapolation) Improve readings with large error (intersection) Extrapolation: in round i use the reading from round i − 1 shift clock value by P, extend error by (k + 2)Pρmax Intersection: intersect readings from round i with extrapolation from i − 1 both correct → result also correct Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  29. 29. Motivation Background Adaptive Clock Sync. Evaluation Summary 16 of 25 slides Extrapolation & Intersection Re-use clock readings from previous round Mask transient clock reading failures (extrapolation) Improve readings with large error (intersection) Extrapolation: in round i use the reading from round i − 1 shift clock value by P, extend error by (k + 2)Pρmax Intersection: intersect readings from round i with extrapolation from i − 1 both correct → result also correct Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  30. 30. Motivation Background Adaptive Clock Sync. Evaluation Summary 16 of 25 slides Extrapolation & Intersection Re-use clock readings from previous round Mask transient clock reading failures (extrapolation) Improve readings with large error (intersection) Extrapolation: in round i use the reading from round i − 1 shift clock value by P, extend error by (k + 2)Pρmax Intersection: intersect readings from round i with extrapolation from i − 1 both correct → result also correct Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  31. 31. Motivation Background Adaptive Clock Sync. Evaluation Summary 17 of 25 slides Evaluation Environment OMNeT++ discrete event based simulator global observer view only possible in simulation TDMA 10 Mbit, half-duplex, shared Ethernet 2 · 105 km/s signal propagation speed 4 hosts: A 10 [m] B 10 [m] C 10 [m] D Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  32. 32. Motivation Background Adaptive Clock Sync. Evaluation Summary 18 of 25 slides Imprecision – cfnV 0 () & cfnV 1 () 60 V0 - 0% V0 - 25% V1 - 1% V0 - 1% V1 - 0% V1 - 25% 50 40 imprecision [µs] 30 20 10 0 25 30 35 40 45 50 55 60 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  33. 33. Motivation Background Adaptive Clock Sync. Evaluation Summary 19 of 25 slides Intersection Gain 0% -host A 1% - host D 0.6 0% - host D 25% - host A 1% - host A 25% - host D 0.5 mean improvement [%] 0.4 0.3 0.2 0.1 0 5 10 15 20 25 30 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  34. 34. Motivation Background Adaptive Clock Sync. Evaluation Summary 19 of 25 slides Intersection Gain 7 0% -host A 1% - host D 0% - host D 25% - host A 6 1% - host A 25% - host D 5 improvement [%] 4 3 2 1 0 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 9 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  35. 35. Motivation Background Adaptive Clock Sync. Evaluation Summary 20 of 25 slides Extrapolation and Omissions 10000 max. sync. error real sync. error 1000 infinity 100 error [µs] 10 1 0.1 0.01 0.001 21.6 21.8 22 22.2 22.4 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  36. 36. Motivation Background Adaptive Clock Sync. Evaluation Summary 20 of 25 slides Extrapolation and Omissions 10000 max. sync. error real sync. error 1000 no infinity 100 (extrapolation) error [µs] 10 1 0.1 0.01 0.001 21.6 21.8 22 22.2 22.4 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  37. 37. Motivation Background Adaptive Clock Sync. Evaluation Summary 21 of 25 slides Asymmetric Permanent Omissions 120 host A host B 100 host C host D max. sync. error [µs] 80 60 40 20 0 4 5 6 7 8 9 10 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  38. 38. Motivation Background Adaptive Clock Sync. Evaluation Summary 21 of 25 slides Asymmetric Permanent Omissions 120 host A - max sync err host B - max sync err 100 host C - max sync err host D - max sync err host A - real sync err real/max sync err [µs] 80 host B - real sync err host C - real sync err host D - real sync err 60 40 20 0 4 5 6 7 8 9 10 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  39. 39. Motivation Background Adaptive Clock Sync. Evaluation Summary 21 of 25 slides Asymmetric Permanent Omissions 120 host C - max sync err host D - max sync err 100 host C - real sync err host D - real sync err real/max sync err [µs] 80 60 40 20 0 4 5 6 7 8 9 10 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  40. 40. Motivation Background Adaptive Clock Sync. Evaluation Summary 22 of 25 slides Crash Failures 120 host A - max. sync. error host A - real sync. error 100 host B - max. sync. error host B - real sync. error host D - max. sync. error 80 host D - real sync. error error [µs] 60 40 20 0 34 36 38 40 42 44 46 48 50 real-time [s] Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  41. 41. Motivation Background Adaptive Clock Sync. Evaluation Summary 23 of 25 slides Summary Need to build critical systems, however: underlying components are not synchronous need to use COTS to cut cost need to use wireless to cut weight We propose Adaptive Internal Clock Synchronization copes with message delays and process failures bounds deviation Ep (T ) from correct clocks in the system propagates the deviation Ep (T ) to upper level applications Applications use the Ep (T ) to adapt their behavior no more synchronization layer crashes the system is safer Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  42. 42. Motivation Background Adaptive Clock Sync. Evaluation Summary 23 of 25 slides Summary Need to build critical systems, however: underlying components are not synchronous need to use COTS to cut cost need to use wireless to cut weight We propose Adaptive Internal Clock Synchronization copes with message delays and process failures bounds deviation Ep (T ) from correct clocks in the system propagates the deviation Ep (T ) to upper level applications Applications use the Ep (T ) to adapt their behavior no more synchronization layer crashes the system is safer Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  43. 43. Motivation Background Adaptive Clock Sync. Evaluation Summary 23 of 25 slides Summary Need to build critical systems, however: underlying components are not synchronous need to use COTS to cut cost need to use wireless to cut weight We propose Adaptive Internal Clock Synchronization copes with message delays and process failures bounds deviation Ep (T ) from correct clocks in the system propagates the deviation Ep (T ) to upper level applications Applications use the Ep (T ) to adapt their behavior no more synchronization layer crashes the system is safer Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  44. 44. Motivation Background Adaptive Clock Sync. Evaluation Summary 24 of 25 slides Thank You! http://wwwse.inf.tu-dresden.de/ Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer
  45. 45. Motivation Background Adaptive Clock Sync. Evaluation Summary 25 of 25 slides References Flaviu Cristian and Christof Fetzer. Probabilistic internal clock synchronization. In Proceedings of the Thirteenth Symposium on Reliable Distributed Systems (SRDS1994), pages 22–31, October 1994. Flaviu Cristian and Christof Fetzer. The timed asynchronous distributed system model. IEEE Trans. on Parallel and Distr. Systems, 10(6):642–657, June 1999. Christof Fetzer and Flaviu Cristian. Building fault-tolerant hardware clocks. In Proceedings of the Seventh IFIP Int. Working Conf. on Dependable Computing for Critical Applications, pages 59–78, San Jose, USA, Jan 1999. Christof Fetzer and Flaviu Cristian. A fail-aware datagram service. In Iain Bate and Alan Burns, editors, IEE Proceedings - Softw. Eng., volume 146, pages 58–74. IEE, April 1999. Christof Fetzer and Flaviu Cristian. Fail-awareness: An approach to construct fail-safe systems. Journal of Real-Time Systems, 24(2):203–238, March 2003. Ute Wappler and Christof Fetzer. Hardware failure virtualization via software encoded processing. In 5th IEEE Int. Conf. on Industrial Inf. (INDIN 2007), volume 2, pages 977–982, June 2007. Jennifer Lundelius Welch and Nancy Lynch. A new fault-tolerant algorithm for clock synchronization. Information and Computing, 77(1):1–36, 1988. Adaptive Internal Clock Synchronization Zbigniew Jerzak, Robert Fach, and Christof Fetzer

×