Self Optimizing transactional data grids for elastic cloud environments

597 views

Published on

Paolo Romano

Published in: Travel, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
597
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Self Optimizing transactional data grids for elastic cloud environments

  1. 1. !"#$%&()*+*,-./01,213(&,1#.41/1.-0*42.$&0. "#12(3.3#&54.",6*0&,)",/2. 71&#&.8&)1,&. ! Cloudviews 2011, Porto, Portugal, Nov. 4 2011
  2. 2. About me•  "#$%&!#()!*+,!-./!01#23&(4#5!6(37&$3%8!.-!9./&!•  9&$&#:+&!#%!,3$%3;<%&)!18$%&/$!=.<2>!?@A1BC?,>!D3$;.(!E$3(:&!FGGHI! –  9"2/.:;<!=%:>.?&5,-.8"2"103@"0.ABCC!•  ?(73%&)!2.-&$$.!#%!?($J%<%.!1<2&3.!KL:(3:.>!D3$;.(!E$3(:&!FGMMI!1./&!3(%&(#J.(#N!2.O&:%$!3(!P+3:+!?!#/!:<&(%N8!3(7.N7&)Q!•  B..)3(#%.!.-!%+&!R*S!BN.<)CK"!*.O&:%!ET<(!FGMGCT<(FGMFI! –  U!3(%&(#J.(#N!2#%(&$!-./!3()<$%8!#()!#:#)&/8!•  B..)3(#%.!.-!%+&!B.$%!V:J.(!A<.CK"!E-#NN!FGMGC-#NN!FGMWI! –  *#(CA<.2&#(!9&$&#:+!(&%P.X!.(!K#($#:J.(#N!"&/.3&$! –  YZ!&[2&%$>!UF!3($J%<J.($>!MF!:.<(%3&$!!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  3. 3. Talk overview•  BN.<)CK"!^7&73&PQ! –  X&8!.#N$! –  ;#:X.<()!.(!K#($#:J.(#N!"&/.3&$! –  2.&$$&$!$.!-#!•  1&N-C.2J/343(!%#($#:J.(#N!)#%#!3)$Q! –  /&%+.).N.3&$!&[2N.&)!$.!-#! –  :#$&!$%<)3&$!•  ^2&(!&$&#:+!_<&$J.($!`!-<%<&!P.X!Cloudviews 2011, Porto, Portugal, Nov. 4 2011
  4. 4. Cloud-TM at a glance "#$%&$()! ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"#$%&!"!()*+! !!!!!!!!!!!!!!!!!,-./01234156!("*+ ! ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&7"7#7"7!("*+ ! !!!!!!!!!!!!!!!!!!89:!;62!("$+ "$*+,%!,**$-.&#%*$)! /0$#1*&)! )6/-/!8/46</=!"#$%&!"!()*+! >0/4!?@<9!ABCB!2/!D6E!ABCF! "$*2$#33)! 40$%5$!.&6*$3#1*&)! >)GH"&*HABBIHJ!K!LMN95OP9!C7A! 3QRSTTUUU75-/@:2479@!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  5. 5. Key Goals,&7&N.2!#!%#($#:J.(#N!)#%#!2N#a./!-.!%+&!BN.<)Q!M]  *.73)3(!#!$3/2N&!#()!3(%<3J7&!2.#//3(!/.)&NQ!   +3)&!:./2N&[3%8!.-!)3$%3;<J.(>!2&$3$%&(:&>!-#<N%C %.N&#(:&!F]  "3(3/343(!#)/3(3$%#J.(!#()!/.(3%.3(!:.$%$Q!   #<%./#%&!&N#$J:!&$.<:&!2.73$3.(3(!;#$&)!.(! #22N3:#J.($!b.1!&_<3&/&(%$!W]  "3(3/34&!.2&#J.(#N!:.$%$!73#!$&N-C%<(3(!   /#[3/34&!&c:3&(:8!#)#2J(!:.($3$%&(:8!/&:+#(3$/$! <2.(!:+#(&$!.-!P.XN.#)!#()!#NN.:#%&)!&$.<:&$!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  6. 6. d#:X.<()!.(!%+&!BN.<)CK"!*.#//3(!*##)3/e]! TRANSACTIONAL MEMORIESBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  7. 7. Transactional Memories!•  K#($#:J.(#N!"&/.3&$!EK"IQ! –  &2N#:&!N.:X$!P3%+!#%./3:!%#($#:J.($!3(!%+&! 2.#//3(!N#(<#&! –  +3)&!#P#8!$8(:+.(34#J.(!3$$<&$!-./!%+&! 2.#//&! •  #7.3)!)&#)N.:X$>!23.3%8!3(7&$3.($>!)&;<3(!(3+%/#&! •  $3/2N&!%.!&#$.(!#;.<%>!7&3-8>!:./2.$&! –  2*)#*$D.4"6"#&)",/.&$.101##"#.1#*31(&,2.!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  8. 8. !to Distributed Transactional Memories... •  ,3$%3;<%&)!K#($#:J.(#N!"&/.3&$!E,K"IQ! –  &[%&()$!K"!#;$%#:J.(!.7&!%+&!;.<()#3&$!.-!#! $3(N&!/#:+3(&Q! •  &(+#(:&!$:#N#;3N3%8! •  )<#;3N3%8!73#!3(C/&/.8!&2N3:#J.(! ! –  /3(3/34&!:.//<(3:#J.(!.7&+&#)!73#Q! •  $2&:<N#J.(! •  ;#%:+3(!:.($3$%&(:8!#:J.($!#%!:.//3%CJ/&! BN.<)!B./2<J(!&[2&%$!.<2!C!1&2%]!FH!FGMM!C!d<[&NN&$!
  9. 9. !to Cloud-TM^2&(C$.<:&!,K"!/3))N&P#&!2.73)3(Q!•  D#(<#&!N&7&N!$<22.%!-.Q! –  .;O&:%C.3&(%&)!)./#3(!/.)&N! –  +3+N8!$:#N#;N&!#;$%#:J.($!!•  AN#$J:!$:#N&C<2!#()!$:#N&C).P(!.-!%+&!,K"!2N#a./Q! –  #<%./#J:!$:#N3(!;#$&)!.(!<$&!)&f(&)!b.1!`!:.$%!:.($%#3(%$! !•  1&N-C.2J/34#J.(!#$!#!2&7#$37&!-&#%<&Q! –  2<$<&!/#[3/</!&c:3&(:8!73#!:.$$CN#8&!$&N-C%<(3(!!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  10. 10. PROGRESSES SO FARBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  11. 11. Main achievements•  V:+3%&:%<&!$2&:3f:#J.(!!•  ,&7&N.2/&(%!.-!2&N3/3(#8!2.%.%82&!•  ?((.7#J7&!%#($#:J.(#N!&2N3:#J.(!$:+&/&$!•  *N#a./!$&N-C%<(3(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  12. 12. Architecture Specification 31$"456*)01+2$.&) /+#+)01+2$.&) !"#$%$&()*+%+,-.) J#M8-?$-N+8):8"B9$/%!$ 0"?"$%>"E&C$%&;&"CC)9;$/%!-$ 0)-?&)=<?+*$ 0"?"$%>"E&C$ 2=O+8?$5&)*$ F27GH2/0$I$J#$423!127$ 2NBC)T+&$ #+"&8P$/%!$ 6Q+8<B9$ 4"NN+&$ /0/%1/1!23$4/3/567$ F27GH2/0$/3/HKL67$ R&"C+A&S$ 7+89:;<&"=>+$0)-?&)=<?+*$ #@A"&+$1&"9-"8B9">$4+C&D$ 6>"-B8$#8">)9;$ 4"9";+&$ 7+89:;<&"=>+$#?&";+$#D-?+C$ /+#+)01+2$.&)>-($%?,".+=$%)@)6"%%,) .-7$".(-) 8.$97$%%,):) !""#$%&()*+&,-.$ ;<!)%-,$=+=$%)BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  13. 13. Preliminary prototype•  M$%!7&$3.(!.-!,#%#!*N#a./!#N&#)8!#7#3N#;N&Q! !"#$%%&&&()*+,-./0+1 !•  ?(%&#J.(g&[%&($3.(!.-!/#3($%&#/!.2&(!$.<:&! 2.O&:%$Q! –  -.:<$!.(!3((.7#J.(!`!#7.3)!&3(7&(J(!%+&!P+&&N! –  /#[3/34&!2.O&:%h$!73$3;3N3%8!`!-#:3N3%#%&!&[2N.3%#J.(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  14. 14. Innovative transactional replication schemes•  1&7&#N!#22.#:+&$!+#7&!;&&(!2<$<&)Q! –  .7&N#2!2.:&$$3(!#()!:.//<(3:#J.(!73#!$2&:<N#J.(! i1*VVFGMG>!?1*VFGMG>!@BVFGMG>!1j1K^9FGMM>19,1FGMMk! –  #$8(:+.(.<$!N&#$&$!%.!&)<:&!:.//<(3:#J.(!.7&+&#)! i"3))N&P#&FGMGk! –  P&#X&!:.($3$%&(:8!/.)&N$! i*9,BFGMMk!•  ,3l&&(%!#22.#:+&$!P3%+!#!$#/&!:.//.(!.#NQ!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  15. 15. Data-grid self-optimization•  1&N-C%<(3(g2&-./#(:&!-.&:#$J(!.-!$&7&#N! 2N#a./!N#8&$! –  1.mP#&!K#($#:J.(#N!"&/.8!N#8&! iB"=MGk>!i*AnVMMk! ! –  9&2N3:#J.(!/#(#&! i"3))N&P#&FGMMk! ! –  =.<2!:.//<(3:#J.(!$8$%&/! i1V1^MGk>!i*&-./#(:&FGMMk>i?B@BMFk! !BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  16. 16. Talk overview•  BN.<)CK"!^7&73&PQ! –  X&8!.#N$! –  ;#:X.<()!.(!K#($#:J.(#N!"&/.3&$! –  2.&$$&$!$.!-#!•  1&N-C.2J/343(!%#($#:J.(#N!)#%#!3)$Q! –  /&%+.).N.3&$!&[2N.&)!$.!-#! –  :#$&!$%<)3&$!•  ^2&(!&$&#:+!_<&$J.($!`!-<%<&!P.X!Cloudviews 2011, Porto, Portugal, Nov. 4 2011
  17. 17. Methodologies explored so far•  V(#N8J:#N!/.)&N3(Q! –  _<&<3(!%+&.8>!/#X.7!2.:&$$&$! –  $%.:+#$J:!%&:+(3_<&$!•  "#:+3(&!N&#(3(Q! –  .lCN3(&!%&:+(3_<&$Q! •  ,&:3$3.(!K&&$>!@&<#N!(&%P.X$>!1<22.%!n&:%.!"#:+3(&! –  .(CN3(&!%&:+(3_<&$!E&3(-.:&/&(%!N&#(3(IQ! •  6Bd!#N.3%+/!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  18. 18. Analytical modeling•  P+3%&!;.[!#22.#:+Q! –  &_<3&$!)&%#3N&)!X(.PN&)&!.-!3(%&(#N!)8(#/3:$!•  ..)!&[%#2.N#J.(!2.P&Q!! –  #NN.P!-.&:#$J(!$8$%&/!;&+#73.!3(!<(&[2N.&)! &3.($!.-!3%$!2##/&%&$h!$2#:&!!!•  /3(3/#N!N&#(3(!J/&Q! –  ;#$3:#NN8!2##/&%&$!3($%#(J#J.(!!!•  :./2N&[!#()!&[2&($37&!%.!)&$3(g7#N3)#%&!"!•  $<;O&:%!%.!<(#7.3)#;N&!#22.[3/#J.(!&.$!"!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  19. 19. Machine learning•  ;N#:X!;.[!#22.#:+Q! –  .;$&7&!3(2<%$>!:.(%&[%!#()!.<%2<%$!.-!#!$8$%&/! –  <$&!$%#J$J:#N!/&%+.)$!%.!3)&(J-8!2#o&($g<N&$!•  ..)!#::<#:8!3(!#N&#)8!&[2N.&)!&3.($!.-!%+&! 2##/&%&$h!$2#:&!!!•  e;<%!2..!&[%#2.N#J.(!2.P&!"!•  N&#(3(!J/&!.P$!&[2.(&(J#NN8!P3%+!(</;&! .-!-&#%<&$Q! –  ;<%!&7&(%<#NN8!.<%2&-./$!#(#N8J:#N!/.)&N$! E%823:#NN8pI!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  20. 20. Hybrid techniques :><EF.-"/./@".G"2/.&$./@"./H&.H&0#42 .KP.!#N%&(#J7&!#22.#:+&$!$.!-#Q! M]  ,373)&C#()C:.(_<&Q! •  V"!-.!P&NNC$2&:3f&)!$<;C:./2.(&(%$! •  "D!-.!$<;C:./2.(&(%$!%+#%!#&Q! –  %..!:./2N&[!%.!/.)&N!&[2N3:3%N8>!.! –  P+.$&!3(%&(#N!)8(#/3:$!#&!.(N8!2#J#NN8!$2&:3f&)! F]  6$&!V"!%.!3(3J#N34&!"D!X(.PN&)&Q! •  &)<:&!N&#(3(!J/&!.-!"D!%&:+(3_<&$! •  :.&:%!V"!<$3(!-&&);#:X!-./!.2&#J.(#N!$8$%&/!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!!
  21. 21. Case studies•  ,8(#/3:!$&N&:J.(!#()!$P3%:+3(!.-!)#%#! &2N3:#J.(!2.%.:.N$Q! –  %.%#N!.)&!;#$&)!&2N3:#J.(!2.%.:.N$!EB#$&!$%<)8!MIQ! •  2<&N8!;#$&)!.(!"#:+3(&!D&#(3(!%&:+(3_<&$! –  $3(N&C/#$%&!7$!/<NJC/#$%&!EB#$&!$%<)8!FIQ! •  +8;3)!"DCV"!$.N<J.(!q!)373)&C&%C3/2&#!•  =.<2!B.//<(3:#J.(!18$%&/!$&N-C.2J/34#J.(Q! –  ;#%:+3(!3(!%.%#N!.)&!2.%.:.N$!EB#$&!$%<)8!WI! •  +8;3)!"DCV"!q!"D!;..%$%#22&)!P3%+!V"!X(.PN&)&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  22. 22. SELF-OPTIMIZING DATA CONSISTENCY PROTOCOLSBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  23. 23. The search for the holy grail %#($#:J.(#N!)#%#! :.($3$%&(:8!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! K.%#N!.)&C;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(! dRB!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  24. 24. The Cloud-TM vision •  low #resources: •  auto-scale up: •  multi-master: •  auto-scale down: - minimum costs - new nodes hired for - minimum costs read-only requests - hi % write •  primary-backup: primary overwhelmed •  switch back to •  primary-backup: - low % write: •  higher scalability primary-backup - low % write: low load on primary primary stands the load : node processing read-only requests : node processing read&write requests •  low traffic •  hi traffic •  hi traffic •  low traffic •  read-dominated •  read-dominated •  write dominated •  read dominated • low conflict •  low conflict •  low conflict • low conflict %&($ /$%$#&()$ %$#&()$ !"#$ !"#$%$#&()$(&*+,*-."+,$ %&($ /$(&*4-$ &)2,3,)-$ !"#$(&*4-$ !"#$ 01)$BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  25. 25. Self-optimizing data replication: key challengesM]  #NN.P!&c:3&(%!$P3%:+!#/.(!/<NJ2N&!&2N3:#J.(! 2.%.:.N$Q! –  #7.3)!;N.:X3(!%#($#:J.(!2.:&$$3(!)<3(! %#($3J.($!F]  )&%&/3(&!%+&!.2J/#N!&2N3:#J.(!$%#%&8! 37&(!%+&!:<&(%!P.XN.#)!:+##:%&3$J:$Q! –  /#:+3(&!N&#(3(!/&%+.)$!E;N#:X!;.[I! –  #(#N8J:#N!/.)&N$!EP+3%&!;.[I! –  +8;3)!#(#N8J:#Ng$%#J$J:#N!#22.#:+&$!E#8!;.[I!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  26. 26. Case studies•  ,8(#/3:!$&N&:J.(!#()!$P3%:+3(!;&%P&&(! &2N3:#J.(!2.%.:.N$Q! –  %.%#N!.)&!;#$&)!&2N3:#J.(!2.%.:.N$!EB#$&!$%<)8!MIQ! •  2<&N8!;#$&)!.(!"#:+3(&!D&#(3(!%&:+(3_<&$! –  KP.!2+#$&!:.//3%!7$!23/#8!;#:X<2!EB#$&!$%<)8!FIQ! •  +8;3)!"DCV"!$.N<J.(!q!)373)&C&%C3/2&#!•  =.<2!B.//<(3:#J.(!18$%&/!$&N-C.2J/34#J.(Q! –  ;#%:+3(!3(!%.%#N!.)&!2.%.:.N$!EB#$&!$%<)8!WI! •  +8;3)!"DCV"!q!"D!;..%$%#22&)!P3%+!V"!X(.PN&)&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  27. 27. ! "#3#!B.<:&3.>!*#.N.!9./#(.>!D<3$!9.)3<&$! VB"g?R?*g61A@?r!MF%+!?(%&(#J.(#N!"3))N&P#&!B.(-&&(:&! E"3))N&P#&!FGMMI! POLYCERT: POLYMORPHIC SELF-OPTIMIZING REPLICATION FOR IN-MEMORY TRANSACTIONAL GRIDSBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  28. 28. Where they fit in the picture 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! K.%#N!.)&C;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(! dRB!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  29. 29. Certification (a.k.a. deferred update)•  V!%#($#:J.(!3$!&[&:<%&)!3()&2&()&(%N8!#%!#!$3(N&! &2N3:#!<(JN!3%$!:.//3%!2+#$&Q! –  /3(3/34&!(&%P.X!%#c:!•  ,3$%3;<%&)!:&Jf:#J.(!3$!<(!%.!)&%&:%!:.(s3:%$!P3%+! %#($#:J.($!&[&:<%&)!:.(:<&(%N8!#%!)3l&&(%! &2N3:#$!•  B&Jf:#J.(!3$!%823:#NN8!/<:+!/.&!N3+%P&3+%!%+#(! -<NN!%#($#:J.(!&[&:<J.(! –  ..)!$:#N#;3N3%8!#N$.!3(!P3%&!3(%&($37&!P.XN.#)$!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  30. 30. Certification•  K+&&!)3l&&(%!#22.#:+&$!3(!N3%&#%<&Q! –  @.(C7.J(!#N.3%+/! –  n.J(!#N.3%+/! –  dRB!•  ^(&!X&8!:.//.(#N3%8Q! –  &N3#(:&!.(!K.%#N!^)&!d.#):#$%!%.!#7.3)!)3$%3;<%&)! )&#)N.:X$! –  K^d!&($<&$!#&&/&(%!.(!)&N37&8!.)&!.-! ;.#):#$%!/&$$#&$!3(!#!(.(C;N.:X3(!-#$+3.(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  31. 31. Classic Replication Protocols•  R.:<$!.(!-<NN!&2N3:#J.(!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! VdC;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(! dRB!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  32. 32. Non-voting K^d!.-!KMh$! K^d!.-!KFh$! &#)!`!P3%&$&%! &#)!`!P3%&$&%! A[&:<J.(! K#($#:J.(!KM!9M! A[&:<J.(! n#N3)#J.(`B.//3%! n#N3)#J.(`V;.%! K#($#:J.(!KF! KM! KF!9F! n#N3)#J.(`B.//3%! n#N3)#J.(`V;.%! KM! KF!9W! + only validation executed at all replicas: high scalability with write intensive workloads - need to send also readset: often very large! ! BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  33. 33. Classic Replication Protocols•  R.:<$!.(!-<NN!&2N3:#J.(!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! VdC;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(! dRB!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  34. 34. Voting !KMh$!K^d! !KMh$!;.#):#$%! E&2I! !E!3*.01I! ! A[&:<J.(! KMh$!! K#($#:J.(!KM! 7#N3)#J.(!9M! ! P#3%!-.!9F! 9Mh$!7.%&! + sends only write-set (much smaller than read-sets normally) - Additional communication phase to disseminate decision (vote) ! BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  35. 35. Classic Replication Protocols•  R.:<$!.(!-<NN!&2N3:#J.(!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! VdC;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(! dRB!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  36. 36. Bloom Filter Certification (BFC)•  6$&!#!$3(N&!K^d!#$!3(!(.(C7.J(>!;<%!&(:.)&!&#)$&%! 3(!#!dN../!fN%&! –  dN../!fN%&$Q! •  $2#:&C&c:3&(%>!2.;#;3N3$J:!)#%#!$%<:%<&!-.!%&$%!/&/;&$+32! •  :./2&$$3.(!3$!#!-<(:J.(!.-!#!E%<(#;N&I!-#N$&!2.$3J7&!#%&!•  V!%#($#:J.(!K!3$!:&Jf&)!$<::&$$-<NN8!.(N8!3-Q! –  (.(&!.-!%+&!3%&/$!P3o&(!;8!:.(:<&(%!%#($#:J.($!3$! 2&$&(%!3(!%+&!dR!<$&)!%.!&(:.)&!Kh$!&#)$&)! –  $%.(N8!&)<:&!(&%P.X!%#c:!#%!%+&!:.$%!.-!(&N33;N&! #;.%!3(:&#$&! •  Mt!-#N$&!2.$3J7&!83&N)$!<2!%.!WG[!:./2&$$3.(!!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  37. 37. BFC vs Voting vs Non-Voting (" !#" !"#$%&()*+,-#"./-0.1+ !#&" *+," !#%" -.," +," !#$" !" (" "()!!!"" "(!!)!!!"" 2)%*+3)1+3()+ Fig. 2. Throughput of three certificationBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM! strategies with different read-set sizes.
  38. 38. Self-optimizing data replication: key challengesM]  #NN.P!&c:3&(%!$P3%:+!#/.(!/<NJ2N&!&2N3:#J.(! 2.%.:.N$Q! –  3&"I*2/",3"!.-!/<NJ2N&!:&Jf:#J.(!$:+&/&$!73#!%+&! *.N8/.+3:!B&Jf:#J.(!E*.N8B&%I!2.%.:.N!F]  )&%&/3(&!%+&!.2J/#N!&2N3:#J.(!$%#%&8! 37&(!%+&!:<&(%!P.XN.#)!:+##:%&3$J:$Q! –  &(J&N8!;#$&)!.(!)13@*,".#"10,*,-.%&:+(3_<&$! •  .lCN3(&Q!)&:3$3.(C%&&$>!(&<#N!(&%P.X>!1n"! •  .(CN3(&Q!&3(-.:&/&(%!N&#(3(!E6BdI! !BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  39. 39. PolyCert•  *.N8/.2+3:!1&N-C^2J/343(!B&Jf:#J.(!!•  B.C&[3$%&(:&!.-!%+&!W!:&Jf:#J.(!$:+&/&$Q! –  &[2N.3%!:.//.(!&N3#(:&!.(!%.%#N!.)&!;.#):#$%!•  "#:+3(&CN&#(3(!%&:+(3_<&$!%.!)&%&/3(&! %+&!.2J/#N!:&Jf:#J.(!$%#%&8!2&! %#($#:J.(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  40. 40. Replication Protocol Selector Oracle•  KP.!3/2N&/&(%#J.($Q! – ^lCN3(&!"#:+3(&!D&#(3(!K&:+(3_<&$! – ^(CN3(&!9&3(-.:&/&(%!D&#(3(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  41. 41. Off-line Machine Learning Techniques•  R.!&#:+!%#($#:J.(Q! –  ,&%&/3(&!$34&!.-!&[:+#(&)!/&$$#&$!-.!&#:+! :&Jf:#J.(!$:+&/&! –  R.&:#$%!Vd!N#%&(:8!-.!&#:+!/&$$#&!$34&]!u&! &7#N<#%&)!$&7&#N!"D!#22.#:+&$Q! •  9&&$$3.(!)&:3$3.(!%&&$!E;&$%!&$<N%$I!! •  @&<#N!(&%P.X$!! •  1<22.%!7&:%.!/#:+3(&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  42. 42. Off-line Machine Learning Techniques•  6$&$!<2!%.!YW!/.(3%.&)!$8$%&/!#o3;<%&$Q!! –  B*6! –  "&/.8!! –  @&%P.X!! –  K3/&C$&3&$!•  9&_<3&$!:./2<%#J.(#N!3(%&($37&!-&#%<&! $&N&:J.(!#()!%#3(3(!2+#$&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  43. 43. On-line Reinforcement Learning•  A#:+!&2N3:#!;<3N)$!.(CN3(&!&[2&:%#J.($!.(!%+&! &P#)$!.-!&#:+!2.%.:.NQ! –  (.!#$$</2J.(!.(!&P#)$h!)3$%3;<J.($!!•  622&!B.(f)&(:&!d.<()!E6BdI!#N.3%+/Q!! –  N3+%P&3+%!#()!2.7#;N8!.2J/#N!$.N<J.(!%.!%+&! &[2N.#J.(C&[2N.3%#J.(!)3N&//#Q! •  )3)!?!%&$%!%+3$!.2J.(!$<c:3&(%N8!3(!%+3$!$:&(#3.v!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  44. 44. On-line Reinforcement Learning•  ,3$J(<3$+&$!P.XN.#)!$:&(#3.!$.N&N8!;#$&)! .(!&#)C$&%h$!$34&! –  &[2.(&(J#N!)3$:&J4#J.(!3(%&7#N$!%.!/3(3/34&! %#3(3(!J/&!•  9&2N3:#$!&[:+#(&!$%#J$J:#N!3(-./#J.(! 2&3.)3:#NN8!%.!;..$%!N&#(3(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  45. 45. Chasing the optimum! (" !#" !"#$%&()*+,-#"./-0.1+ !#&" *+," -.," !#%" +," /0123,-" !#$" /4" !" (" "()!!!"" "(!!)!!!"" 2)%*+3)1+3()+ Fig. 5. Normalized throughput of the adaptive and non-adaptive protocols (Bank benchmark).BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!transactions, thus allowing us to focus on the performance of the transactions that re-
  46. 46. !and beating it! (#%" (#$" !"#$%&()*+,-#"./-0.1+ (" )*" !#" +," !#&" -*." +/01-*." !#%" !#$" !"Fig. 8. Normalized throughput of the adaptive and VC protocols. NVC and BFC not reported asBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!they caused the collapse of the GCS layer. (STMBench7 benchmark)
  47. 47. Case studies•  ,8(#/3:!$&N&:J.(!#()!$P3%:+3(!;&%P&&(! &2N3:#J.(!2.%.:.N$Q! –  %.%#N!.)&!;#$&)!&2N3:#J.(!2.%.:.N$!EB#$&!$%<)8!MIQ! •  2<&N8!;#$&)!.(!"#:+3(&!D&#(3(!%&:+(3_<&$! –  $3(N&C/#$%&!7$!/<NJC/#$%&!EB#$&!$%<)8!FIQ! •  +8;3)!"DCV"!$.N<J.(!q!)373)&C#()C:.(_<&!•  =.<2!B.//<(3:#J.(!18$%&/!$&N-C.2J/34#J.(Q! –  ;#%:+3(!3(!%.%#N!.)&!2.%.:.N$!EB#$&!$%<)8!WI! •  +8;3)!"DCV"!q!"D!;..%$%#22&)!P3%+!V"!X(.PN&)&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  48. 48. ,3&.!,3).(#>!1&;#$J#(.!*&N<$.>!*#.N.!9./#(.!#()!R#(:&$:.!b<#N3#>!! 1&N-C%<(3(!&2N3:#J.(!.-!&N#$J:!3(C/&/.8!%#($#:J.(#N!)#%#!2N#a./$>! !?@A1BC?,!K&:]!9&2]!FYgFGMM>!!"#8!FGMM]! SINGLE VS MULTI-MASTERBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  49. 49. Classic Replication Protocols•  R.:<$!.(!-<NN!&2N3:#J.(!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! K.%#N!.)&!;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  50. 50. Single Master•  u3%&!%#($#:J.($!#&!&[&:<%&)!&(J&N8!3(!#!$3(N&! &2N3:#!E%+&!23/#8I!•  ?-!#!P3%&!%#($#:J.(!3$!&#)8!%.!:.//3%>!:..)3(#J.(! 3$!&_<3&)!%.!<2)#%&!#NN!%+&!.%+&!&2N3:#$!E;#:X<2$I]!•  9&#)!%#($#:J.($!:#(!;&!&[&:<%&)!.(!;#:X<2!&2N3:#$]!•  ;&.4*2/0*G5/"4.4"14#&3J2.•  ;&.4*2/0*G5/"4.3&&04*,1(&,.450*,-.3&))*/.•  K@0&5-@5/.&$.H0*/"./I2.4&"2,L/.231#".5.H*/@.,5)G"0.&$.,&4"2.BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  51. 51. Classic Replication Protocols•  R.:<$!.(!-<NN!&2N3:#J.(!2.%.:.N$! 13(N&!/#$%&!! "<NJ!/#$%&!! E23/#8C;#:X<2I! K.%#N!.)&!;#$&)! F*BC;#$&)! B&Jf:#J.(! 1%#%&!/#:+3(&!&2N3:#J.(! @.(C7.J(! n.J(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  52. 52. 2PC-based replication•  Transactions executed at all nodes w/o coordination till commit time•  Acquire atomically locks at all nodes using two phase commit (2PC): •  2PC materializes conflicts among concurrent remote transactions generating: DISTRIBUTED DEADLOCKS + good scalability at low conflict - thrashes at high conflict ! BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  53. 53. Goals•  V<%.(./3:#NN8!$&N&:%!%+&!;&$%!$<3%&)!2.%.:.N! %+#%! –  "3(3/34&$!%#($#:J.($w!$&73:&!J/&! –  "#[3/34&$!#:+3&7#;N&!%+.<+2<%!•  V<%./#%&!&N#$J:!$:#N3(! –  1:#N&!<2!3-!%+&!$8$%&/!(&&)$!/.&!:./2<%#J.(#N! 2.P&! –  1:#N&!).P(!3-!%+&!$8$%&/!3$!.7&$34&)!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  54. 54. Self-optimizing data replication: key challengesM]  #NN.P!&c:3&(%!$P3%:+!#/.(!/<NJ2N&!&2N3:#J.(! 2.%.:.N$Q! –  +&&!:.&[3$%&(:&!.-!%+&!F!$:+&/&$!3$!3/2.$$3;N&Q! •  )&$3(!.-!#(!(.(C;N.:X3(!2.%.:.N!$P3%:+3(!$%#%&8! !F]  )&%&/3(&!%+&!.2J/#N!&2N3:#J.(!$%#%&8! 37&(!%+&!:<&(%!P.XN.#)!:+##:%&3$J:$Q! –  #(#N8J:#N!/.)&N$!.-!&l&:%$!.-!)#%#!:.(%&(J.(! –  /#:+3(&!N&#(3(!/&%+.)$!%.!2&)3:%!+PC)&2&()#(%! N#%&(:3&$! •  B*6!&[&:<J.(!J/&>!(&%P.X!9KK!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  55. 55. Key Technical Problem•  x.P!%.!-.&:#$%Q! –  *&-./#(:&$!.-!2.%.:.N!d!P+3N&!<((3(! 2.%.:.N!Vv! –  *&-./#(:&$!P3%+!r!(.)&$!P+3N&!<((3(!.(!j! (.)&$v!37&(!%+#%!&2N3:#J.(!2.%.:.Ng$:#N&!:+#(&$!#l&:%Q! –  K+&!%#($#:J.(!:.(s3:%!2.;#;3N3%8!! –  K+&!%#($2.%!N#8&!N#%&(:8!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  56. 56. MethodologyT.3(%!<$#&!.-!#(#N8J:#N!/.)&N3(!#()!/#:+3(&!N&#(3(!%&:+(3_<&$Q! –  #(#N8J:#N!/.)&N!.-!&2N3:#J.(!#N.3%+/! )8(#/3:$Q! •  N.:X!:.(%&(J.(>!)3$%3;<%&)!)&#)N.:X!2.;#;3N3%8! •  /&$$#&!&[:+#(&!2#o&(! –  /#:+3(&!N&#(3(!%.!-.&:#$%!2&-./#(:&!.-! .<2!:.//<(3:#J.(!N#8&Q! •  9KK!#$!#!-<(:J.(!.-!/$!$34&>!%+.<+2<%>!y(.)&$!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  57. 57. Analytical Model•  ,3$%3;<%&)!N.:X!:.(%&(J.(!)8(#/3:$!:#2%<&)!73#!#(! #(#N8J:#N!/.)&NQ! –  %+&!&2N3:#J.(!#N.3%+/$w!;&+#73.!3$!-<NN8!$2&:3f&)! –  3%!3$!2.$$3;N&!%.!/#%+&/#J:#NN8!/.)&N!%+&/! !!!!!!45).!*+6!17*.10528)9:1;+.1.!5.<2157*.!0=12.*=9>1?1•  z&8!/&%+.).N.3&$Q! –  "&#(!7#N<&!#(#N8$3$!`!b<&<3(!%+&.8!•  9&N8!.(!"#:+3(&!D&#(3(!%.!-.&:#$%!+#)P#&! )&2&()&(%!/&%3:$>!3(!2#J:<N#!(&%P.X!N#%&(:3&$e!!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  58. 58. Machine learning techniques•  9&$.<:&!73%<#N34#J.(!/#X&$!/#%+&/#J:#N! /.)&N3(!<(-&#$3;N&Q! –  @.!X(.PN&)&!.-!#:%<#N!N.#)!! –  @.!X(.PN&)&!.-!#:%<#N!2+8$3:#N!&$.<:&$!!•  K#($2.%!N#8&!N#%&(:8!E9KKI!.-!%+&!%P.! 2.%.:.N$!2&)3:%&)!73#!)&:3$3.(!%&&! &&$$.$! !BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  59. 59. Inputs for the ML•  @</;&!.-!(.)&$!•  9KK!3(!%+&!:<&(%!:.(f<#J.(!•  134&!.-!&[:+#(&)!/&$$#&$!•  K+.<+2<%!.-!%+&!:<&(%!:.(f<#J.(! –  M,J,&H,NNN. –  =<&$$&)!<$3(!%+&!#(#N8J:#N!/.)&N!E/.&!(&[%I!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  60. 60. Statistical Model Accuracy •  B.&N#J.(!;&%P&&(!G]{Z!#()!G]{H! •  9&N#J7&!&.!;&%P&&(!G]M{!#()!G]FF!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  61. 61. Models Coupling E,1#D(31#.)&4"#!-.&:#$%$!%+&!41/1.-0*4. /@0&5-@5/!%#X3(!#$!3(2<%!%+&!8KK!3(!%+&!%#&%! :.(f<#J.(]! . O13@*,".#"10,*,-.-.&:#$%$!%+&!8KK!%#X3(!#$!3(2<%! %+&!41/1.-0*4./@0&5-@5/!3(!%+&!%#&%! :.(f<#J.(!!!!R3[&)!2.3(%!$.N<J.(!-.<()!<$3(!&:<$3.(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  62. 62. Global Model Accuracy 20000 2PC-est-low conflict 2PC-low conflict 2PC-est-high conflict 2PC-high conflict PB-est-low conflict PB-low conflict 15000 Throughput (Tx/sec) 10000 5000 0 4 5 6 7 8 9 10 # NodesBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  63. 63. !and now in action! 7000 ’th.txt’ u 1:2 6500 6000 5500 throughput (tx/sec) 5000 4500 4000 3500 3000 PQR. T:UT. PQR. 2500 =Q;SP:=K. =Q;SP:=K. =Q;SP:=K. 2000 0 100 200 300 400 500 600 700 800 900 time (sec)BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  64. 64. Case studies•  ,8(#/3:!$&N&:J.(!#()!$P3%:+3(!;&%P&&(! &2N3:#J.(!2.%.:.N$Q! –  %.%#N!.)&!;#$&)!&2N3:#J.(!2.%.:.N$!EB#$&!$%<)8!MIQ! •  2<&N8!;#$&)!.(!"#:+3(&!D&#(3(!%&:+(3_<&$! –  $3(N&C/#$%&!7$!/<NJC/#$%&!EB#$&!$%<)8!FIQ! •  +8;3)!"DCV"!$.N<J.(!q!)373)&C#()C:.(_<&!•  =.<2!B.//<(3:#J.(!18$%&/!$&N-C.2J/34#J.(Q! –  ;#%:+3(!3(!%.%#N!.)&!2.%.:.N$!EB#$&!$%<)8!WI! •  +8;3)!"DCV"!q!"D!;..%$%#22&)!P3%+!V"!X(.PN&)&!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  65. 65. *#.N.!9./#(.!#()!"#o&.!D&.(&|! @0)A-.+78761B5.(!8761871C*.5)1D=,0=1B=*5,(52.1E=*.*(*)213851F75)9G(5)1 H*,0))876157,1I087A*=(0/07.1J05=78761 ?AAA!?(%&(#J.(#N!B.(-&&(:&!.(!B./2<J(>!@&%P.X3(!#()! B.//<(3:#J.($!E?B@BwMFI>!T#(]!FGMF! ! BATCHING IN TOTAL ORDER BROADCAST PROTOCOLSBN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  66. 66. Sequencer based TOB (STOB)•  K.%#N!.)&!;.#):#$%!EK^dI!#N.3%+/$!&N8!.(! #!$2&:3#N!2.:&$$!%.!&($<&!%.%#N!.)&Q! $&_<&(:&! #$$3($! /! %.%#N!.)&! *F! $&_! $&_! /! *W! K^dE/I! /&$$#&! !!!!/&$$#&! )3l<$3.(! !!!!.)&3(!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  67. 67. Batching in STOB protocols•  1K^d!+#7&!%+&.&J:#NN8!.2J/#N!N#%&(:8Q! –  F!:.//]!$%&2$>!3()&2&()&(%N8!.-!%+&!(</;&!.-!2.:&$$&$!•  e;<%!$&_<&(:&!;&:./&$!%+&!;.oN&(&:X!#%!+3+! %+.<+2<%!•  d#%:+3(!#%!%+&!$&_<&(:&!2.:&$$Q! –  P#3%!-.!$&7&#N!/$$!#()!.)&!%+&/!#N%.&%+&Q! •  #/.J4&!$&_<&(:3(!:.$%!#:.$$!/<NJ2N&!/&$$#&$! –  .2J/#N!P#3J(!J/&!)&2&()$!.(!/&$$#&!#37#N!#%&Q! •  7&8!&l&:J7&!#%!+3+!%+.<+2<%e! •  7&8!;#)!#%!N.P!%+.<+2<%p!BN.<)73&P$!FGMM>!*.%.>!*.%<#N>!@.7]!U!FGMM!
  68. 68. Analytical model •  6$3(!_<&<3(!%+&.8!#</&(%$!P&!:#(! )&%&/3(&!%+&!.2J/#N!;#%:+3(!J/&>!;}>!#$!#! -<(:J.(!.-!%+&!:<&(%!N.#)>!/Q1ueue: linearly and has a vertical asymptote at m = m∗ .  2 1, if m Tadd σ 2 4σ 2 +2Tadd σ 4 (1)   2 + 1 2 2    ). We b∗ (m) = 2m−σ−2mTadd σ (7)  σ−2mT σ+ ,b, m)   2(σ+2m(Tadd σ−1))2   add (2σT −1)2 (1+2σT )σ 2 add add   Tadd σ 2 2 4σ 2 +2Tadd σ 4 if 2 + 1 2 2 m m∗ (2) BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!l rate B. Determining the Model’s Parametersusing
  69. 69. 60Optimal B Avg Msg. Arriv 4000 40 20 Model Accuracy 2000 0 0 2000 4000 6000 8000 10000 12000 14000 0 2 Average Msg. Arrival Rate (msgs/sec) 100 Figure 4. Traffi Exaustive Manual Tuning Analytical Model .)N!()$J/#%$! .2J/#N!;#%:+3(!7#N!Optimal Batching Value #%!0/)3/5!N.#)e! ! 100 Latency (msec) 10 70G#)F. ;5.(!8761 10 +7,0=02G/5G*71(5+2021 292.0/1872.5;8)8.91 1 1 msgs/sec 0 2000 4000 6000 8000 10000 12000 14000 10000 Average Msg. Arrival Rate (msgs/sec) 5000 BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM! Figure 3. Validating the accuracy of the analytical model. 0 16 1
  70. 70. 2000A 0 0 2 4 6 8 10 12 14 16 18 20 22 24 Peak period analysis Hour of the day (3 Sept. 2010) Figure 4. Traffic at the FenixEDU system (Sept. 3 2010) 9#/2C2!`!#/2C).P(! %#($3J.(!%+.+!%+! 100 02.;N/#J:5!##$Q!Latency (msec) C  #/2C2!3$!$c:3(%N8! $+.%Q! 10 !  $8$%/!0$%N$5!;%! :.7$! 1 C  #/2C).P(!3$!N.(Q! !  !!msgs/sec 10000 5000 0 16 17 18 19 20 21 22 Hour of the day Figure 5. Self-tuning based on analytical model BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  71. 71. What about a pure ML approach?•  *.;N/Q! –  D!%:+(3_$!()!%.![2N.!)3l(%!$.NJ.($! E;#%:+3(!7#N$I!%.!3)(J-8!.2J/#N!.(Q! •  N.P!N.#)Q!$N$$!#))3J.(#N!N#%(:8! ! •  /)3/C+3+!N.#)Q!3($c:3(%!;#%:+3(!7#N$!N#)! 78!#23)N8!%.!3($%#;3N3%8!#()!%+#$+3(!!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  72. 72. Combining the two approachesM]  ?(3J#N34!Dh$!X(.PN)!P3%+!%+!2)3:J.($! .-!%+!#(#N8J:#N!/.)NQ! –  ):!-_(:8!.-!.;73.$N8!P.(![2N.#J.($!F]  D%!D!2)#%!%+!3(3J#N!P#)!7#N$Q! –  :.:%!/.)Nh$!2)3:J.(!.$![2N.3J(! -);#:X!-./!%+!$8$%/!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  73. 73. Combining the two approaches lytical Modelry ex- 100 Model+RL ent of Latency (msec)ired to whole 10ed thel Core 1 e timeder of msgs/sec 10000 5000 hough 0 etailed 16 17 18 19 20 21 22ith the Hour of the day above Figure 6. Self-tuning combining RL and analytical model erage) BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM! ntiallygeneric the reward of each possible decision, and lowers it as more
  74. 74. Talk structure•  BN.)CK!^773PQ! –  X8!.#N$! –  ;#:X.()!.(!K#($#:J.(#N!/.3$! –  2.$$$!$.!-#!•  1N-C.2J/343(!%#($#:J.(#N!)#%#!3)$Q! –  $N-C.2J/343(!/%+.).N.3$![2N.)!$.!-#! –  :#$!$%)3$!•  ^2(!$#:+!_$J.($!`!-%!P.X!Cloudviews 2011, Porto, Portugal, Nov. 4 2011
  75. 75. Open research issues•  x.N3$J:!#22.#:+$!%.!$N-C.2J/34#J.(Q! –  ()$%#()!l:%$!.-!$N-C.2J/343(!/NJ2N! /%#NN8!)2()(%!N#8$!!•  b.1C#P#!2.#//3(!2##)3/$Q! –  /%+.).N.3$!#()!%..N$!%.!#NN.P!2.73)$!%.!#$$$$! -#$3;3N3%8!.-!-NfNN3(!1DV$!•  x3+N8!$:#N#;N!)#%#!:.($3$%(:8!/.)N$Q! –  ;8.()!7(%#N!:.($3$%(:8!!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  76. 76. Future work•  R.:$!.(!N#$J:C$:#N3(!X23(!3(%.!#::.(%! )#%#!3)$!)8(#/3:$Q! –  :.($3$%(:8!:.$%$!%#($#:J.(!:.(s3:%$!•  1%)8!l:%$!.-!$N-C%(3(!/NJ2N!/%#NN8! )2()(%!N#8$!.-!%+!)#%#!3)!•  x3+N8!$:#N#;N!_#$3C$3#N34#;N!2.%.:.N$!!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  77. 77. THANKS FOR THE ATTENTIONBN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  78. 78. References (I)iB=MGk!*]!,3!1#(4.!d]!B3:3#(3!R]!b#N3#!9]!*#N/33!#()!*#.N.!9./#(.!V(#N8J:#N!.)NN3(!.-!B.//3%CK3/CD.:X3(!VN.3%+/$!-.!1.mP#!K#($#:J.(#N!/.3$!*.:]!WY%+!?(%(#J.(#N!B./2%!#$/(%!=.2!B.(-(:!EB=I!^N#().!RN.3)#!B./2%!#$/(%!=.2!,:/;!FGMG!!i*AnVMMk!*]!,3!1#(4.!d]!B3:3#(3!R]!b#N3#!9]!*#N/33!#()!*#.N.!9./#(.!^(!%+!V(#N8J:#N!.)N3(!.-!B.(:(:8!B.(%.N!VN.3%+/$!-.!1.mP#!K#($#:J.(#N!/.3$Q!%+!B#$!.-!B.//3%CK3/CD.:X3(!AN$73!*-./#(:!A7#N#J.(!T.(#N!%.!#22#!!i3))NP#FGMGk!@]!B#7#N+.!*#.N.!9./#(.!#()!D]!9.)3$!V$8(:+.(.$!D#$C;#$)!92N3:#J.(!.-!1.mP#!K#($#:J.(#N!/.8!*.:)3($!.-!%+!VBg?R?*g61A@?r!MM%+!3))NP#!B.(-(:!E3))NP#I!d#(#N.!?()3#!VB!*$$!@.7/;!FGMG!!i3))NP#FGMMk!*.N8B%Q!*.N8/.2+3:!1N-C^2J/343(!92N3:#J.(!-.!?(C/.8!K#($#:J.(#N!=3)$!]!B.:3.!*]!9./#(.!#()!D]!9.)3$!VBg?R?*g61A@?r!MF%+!?(%(#J.(#N!3))NP#!B.(-(:!E3))NP#!FGMMI!!!i1*VVFGMGk!*#.N.!9./#(.!9]!*#N/33!R]!b#N3#!@]!B#7#N+.!#()!D]!9.)3$!^(!12:N#J7!92N3:#J.(!.-!K#($#:J.(#N!18$%/$!Ed3-!V((.(:/(%I!*.:]!FF()!VB!18/2.$3/!.(!*##NNN3$/!3(!VN.3%+/$!#()!V:+3%:%$!E1*VVI!1#(%.3(3!=:!VB!*$$!T(!FGMG!!i?1*VFGMGk!*#.N.!9./#(.!9]!*#N/33!R]!b#N3#!@]!B#7#N+.!#()!D]!9.)3$!V(!^2J/#N!12:N#J7!K#($#:J.(#N!92N3:#J.(!*.%.:.N!*.:]!H%+!?AAA!?(%(#J.(#N!18/2.$3/!.(!*##NNN!#()!,3$%3;%)!*.:$$3(!P3%+!V22N3:#J.($!E?1*VI!K#3P#(!K#323!?AAA!B./2%!1.:3%8!*$$!12%/;!FGMG!!i@BVFGMGk!9]!*#N/33!*#.N.!9./#(.!#()!R]!b#N3#!V==9^Q!d..$J(!1K!92N3:#J.(!73#!V$$37N8!^2J/3$J:!K#($#:J.(!*.:$$3(!*.:]!{%+!?AAA!?(%(#J.(#N!18/2.$3/!.(!@%P.X!B./2J(!#()!V22N3:#J.($!E@BVI!B#/;3)!#$$#:+$$%$!61V!?AAA!B./2%!1.:3%8!*$$!TN8!FGMG!!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!
  79. 79. References (II)i1j1K^9FGMMk!@]!B#7#N+.!*#.N.!9./#(.!#()!D]!9.)3$!1B%Q!12:N#J7!BJf:#J.(!3(!92N3:#%)!1.mP#!K#($#:J.(#N!/.3$!*.:)3($!.-!%+!U%+!V((#N!?(%(#J.(#N!18$%/$!#()!1%.#!B.(-(:!E1j1K^9!FGMMI!x#3-#!?$#N!T(!FGMM]!!i19,1FGMMk!9]!*#N/33!R]!b#N3#!#()!*#.N.!9./#(.!^1V9AQ!^22.%(3$J:!12:N#J.(!3(!V:J7N8!9A2N3:#%)!K#($#:J.(#N!18$%/$!E1+.%!*#2I!K+!WG%+!?AAA!18/2.$3/!.(!9N3#;N!,3$%3;%)!18$%/$!E19,1!FGMMI!#)3)!12#3(!^:%]!FGMM!!i1V1^MGk!]!B.:3.!*#.N.!9./#(.!#()!D]!9.)3$!V!#:+3(!D#(3(!V22.#:+!%.!*-./#(:!*)3:J.(!.-!K.%#N!^)!d.#):#$%!*.%.:.N$!!*.:]!U%+!?AAA!?(%(#J.(#N!B.(-(:!.(!1N-CV)#2J7!#()!1N-C^#(343(!18$%/$!E1V1^I!d)#2$%!x(#8!?AAA!B./2%!1.:3%8!*$$!12%/;!FGMG!!i*-./#(:FGMMk!*#.N.!9./#(.!#()!]!D.(|!*.$%Q!1N-C%(3(!d#%:+3(!3(!K.%#N!^)!d.#):#$%!*.%.:.N$!73#!V(#N8J:#N!.)NN3(!#()!93(-.:/(%!D#(3(!VB!*-./#(:!A7#N#J.(!973P!%.!#22#!E#N$.!2$(%)!#%!?R?*!*-./#(:!FGMM!18/2.$3/I!!i?B@BMFk!*#.N.!9./#(.!#()!]!D.(|!1N-C%(3(!d#%:+3(!3(!K.%#N!^)!d.#):#$%!*.%.:.N$!73#!V(#N8J:#N!.)NN3(!#()!93(-.:/(%!D#(3(!?AAA!?(%(#J.(#N!B.(-(:!.(!B./2J(!@%P.X3(!#()!B.//(3:#J.($!@%P.X!VN.3%+/!`!*-./#(:!A7#N#J.(!18/2.$3/!E?B@BwMFI!T#(]!FGMF!!i*9,BFGMMk!A[2N.3J(!K.%#N!^)!NJ:#$%!3(!u#XN8!B.($3$%(%!K#($#:J.(#N!B#:+$!*]!937.!]!B.:3.!*]!9./#(.!#()!D]!9.)3$!*.:]!?AAA!MS%+!*#:3f:!93/!?(%(#J.(#N!18/2.$3/!.(!,2()#;N!B./2J(!E*9,BhMMI!BN.)73P$!FGMM!*.%.!*.%#N!@.7]!U!FGMM!

×