Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Quiver on the Edge: Consistent Scalable Edge Services ASAD SAMAR August 2006 heptF of ile™tri™—l —nd gomputer ingineering g—rnegie wellon …niversity €itts˜urghD €e ISPIQ Submitted in partial ful
  2. 2. llment of the requirements for the degree of Doctor of Philosophy. Thesis committeehrF vujo f—uer @g—rnegie wellon …niversityA€rofF w—uri™e rerlihy @frown …niversityA€rofF fru™e w—ggs @g—rnegie wellon …niversityA€rofF wi™h—el uF ‚eiter @g—rnegie wellon …niversityAD gh—ir€rofF ghenxi ‡—ng @g—rnegie wellon …niversityA 2006 Asad Samar c
  3. 3. 2 ¡ Quiver on the Edge: Consistent Scalable Edge Services
  4. 4. To Tehniat, my wife, my friend, my inspiration
  5. 5. 4 ¡ Quiver on the Edge: Consistent Scalable Edge Services
  6. 6. Abstractrosting dyn—mi™ we˜ servi™es through proxies pl—™ed —t the edge of the snEternet is —n up™oming trend th—t h—s the potenti—l to s™—le these servi™es to— very l—rge num˜er of geogr—phi™—lly distri˜uted ™lientsF roweverD providEing ™onsistent —™™ess to sh—red mut—˜le o˜je™ts th—t m—ke up the servi™eDtoler—ting mis˜eh—ving proxiesD —nd h—ndling proxy dis™onne™tionsD whilestill —™hieving the s™—l—˜ility —nd perform—n™e expe™ted from su™h —n —r™hiEte™ture is — signi™—nt ™h—llengeF „his dissert—tion presents uiverD — distri˜uted o˜je™t system th—t supEports ™onsistent @seri—liz—˜le or stri™tly seri—liz—˜leA oper—tions on sh—redo˜je™ts ˜y servi™e proxies in — wideE—re— settingD while redu™ing the ™lientEper™eived l—ten™yF ‡e —lso present extensions to uiver th—t dete™t ™omproEmised proxies —ttempting to viol—te uiver9s ™onsisten™y propertiesD —llowproxies to dis™onne™t —nd e™iently re™onne™t without del—ying oper—tionsfrom ™onne™ted proxiesD —nd optimize o˜je™t —™™ess times ˜y restru™turinguiver9s ™ommuni™—tion network —™™ording to the worklo—dF uiver proxies —re —rr—nged in — ’lo™—tionE—w—re4 rooted treeF sn orderto perform —n upd—te or — multiEo˜je™t oper—tion involving ™ert—in servi™eo˜je™tsD — uiver proxy migrates those o˜je™ts through the treeD to itselfFy˜je™t migr—tions for oper—tion pro™essing ensures — seri—l exe™ution of upEd—te —nd multiEo˜je™t oper—tions involving the s—me o˜je™tsD —nd en—˜lesuiver to —™hieve the desired ™onsisten™y sem—nti™sD while optimizing forthe @typi™—llyD more frequentA singleEo˜je™t re—d oper—tionsF purthermoreDwhen oper—tions involving —n o˜je™t exhi˜it geogr—phi™ lo™—lity|eFgFD durEing ˜usiness hours on one ™ontinent @—nd nonE˜usiness hours on othersA|the S
  7. 7. 6 ¡ Quiver on the Edge: Consistent Scalable Edge Servicesperform—n™e of these oper—tions ˜enets from the o˜je™t h—ving ˜een miEgr—ted to — ne—r˜y proxyF yther worklo—ds ˜enet from uiver dispersingthe ™ompute lo—d —™ross the proxies performing oper—tionsD —nd s—ving the™osts of tr—nsmitting oper—tion p—r—meters over the ‡ex when these —rel—rgeF qu—r—nteed ™onsisten™y sem—nti™s for sh—red o˜je™ts vi— migr—tionsthrough — tree of proxies requires th—t the proxies —re wellE˜eh—vedD iFeFDfollow the proto™ol spe™i™—tionsF ‡eD thereforeD present —n extension touiver th—t rel—xes this —ssumption ˜y —llowing honest proxies to e™ientlydete™t — mis˜eh—ving proxy th—t —ttempts to ™ompromise the ™onsisten™y ofo˜je™t —™™essesF pin—llyD this dissert—tion —lso dis™usses extensions to m—n—ge the rootedtree @—n overl—yA th—t ™onne™ts the uiver proxiesD in order to improve theover—ll servi™e —v—il—˜ility —nd perform—n™eF sn p—rti™ul—rD we des™ri˜e — disEtri˜uted —lgorithm th—t ™onstru™ts — f—ultEtoler—nt network on top of thetreeD —llowing proxies to e™iently re™onne™t to the prim—ry p—rtition @thep—rtition ™ont—ining the rootAD in ™—se of proxy or link f—iluresF „his efE™ient re™onne™tion —lgorithm redu™es the ’downEtime4 of uiver proxieswhile —voiding some ™entr—l point of reentry @eFgFD the rootA from ˜eingoverlo—ded due to frequent re™onne™t requestsF ‡e —lso dis™uss extensionsth—t heuristi™—lly restru™ture the tree to ˜ring the proxies th—t frequentlyperform oper—tions involving the s—me o˜je™ts ™lose to e—™h otherD gu—r—nEteeing —n O@log nA @for n ™onne™ted proxiesA —mortized o˜je™t —™™ess ™ost for—ny worklo—dF „his dissert—tion det—ils the proto™ols for implementing ™onsistent o˜je™toper—tionsY for —™™ommod—ting the dyn—mi™ —dditionD involunt—ry dis™onEne™tion —nd volunt—ry dep—rture of uiver proxiesY for dete™ting mis˜eh—vEing proxiesY for the ™onstru™tion of — f—ultEtoler—nt network over the treeY—nd for restru™turing the tree —™™ording to the worklo—d to redu™e —™™ess™ostsF „hese —lgorithms —re ev—lu—ted using — ™om˜in—tion of simul—tions—nd experiments performed on €l—netv—˜ —nd isol—ted lo™—l ™lustersF
  8. 8. Acknowledgementss would like to th—nk my —dvisorD wike ‚eiterD for his helpD support —nden™our—gement through the ye—rsF „he extent of wh—t he h—s t—ught mere—™hes mu™h further th—n just rese—r™hF s will —lw—ys ˜e inde˜ted to himF„h—nks wike3 s would —lso like to th—nk ghenxi ‡—ng for —ll the enlightening dis™usEsions we h—ve h—d over the ye—rsF wy st—y —t g—rnegie wellon would noth—ve ˜een —s ple—sur—˜le —s it w—sD h—d it not ˜een for friends like elin—Dplorin —nd gh—rles|who give gre—t —dvi™e on everything from new rese—r™hide—s to life in €itts˜urgh|—nd ƒ™ott q—rriss who is —lw—ys there to lend —h—ndF s do not h—ve the words to express my —ppre™i—tion for my wife „ehni—tDwho s h—ve so needed —nd who h—s —lw—ys provided her unf—ltering love —ndsupportF ƒhe h—s ˜een —ll s ™ould ever —sk forF s would —lso like to th—nk myson ƒh—hv—izD who does not yet know why d—d dis—ppe—rs for d—ys on endD˜ut he ™ert—inly misses me —nd h—s ˜een —s p—tient —s you ™—n expe™t from— two ye—r oldF pin—llyD s would like to express gre—t gr—titude to my p—rents who h—ves—™ri™ed so mu™h over the ye—rs for my edu™—tionF „heir seless devotionDuntiring supportD worldly —dvi™e —nd just pure h—rd work h—s en—˜led meto re—™h this st—ge in my lifeF r—d s ˜een with themD —s s wished —nd theyso deservedD this thesis would not h—ve existedF ‡h—tever its qu—lityD it isindeedD — poor su˜stituteF U
  9. 9. 8 ¡ Quiver on the Edge: Consistent Scalable Edge Services
  10. 10. Contentspigures IQ„—˜les ISI sntrodu™tion I IFI gonsistent o˜je™t —™™ess F F F F F F F F F F F F F F F F F F F F F P IFP hete™ting mis˜eh—ving proxies F F F F F F F F F F F F F F F F F R IFQ ‚e™overing from p—rtitions F F F F F F F F F F F F F F F F F F F F T IFR ‚estru™turing for perform—n™e F F F F F F F F F F F F F F F F F F V IFS ƒtru™ture of this do™ument F F F F F F F F F F F F F F F F F F F IHP gonsistent y˜je™t ƒh—ring IQ PFI ‚el—ted work F F F F F F F F F F F F F F F F F F F F F F F F F F F IQ PFP ƒystem model —nd go—ls F F F F F F F F F F F F F F F F F F F F F IR PFQ y˜je™t m—n—gement F F F F F F F F F F F F F F F F F F F F F F F IT PFQFI distQ —˜str—™tion F F F F F F F F F F F F F F F F F F F F F IT PFQFP distQ implement—tion F F F F F F F F F F F F F F F F F F IU PFQFQ wigr—ting one o˜je™t F F F F F F F F F F F F F F F F F F F IV PFQFR y˜je™t dependen™ies F F F F F F F F F F F F F F F F F F F PH PFR …pd—te —nd multiEo˜je™t oper—tions F F F F F F F F F F F F F F PI PFRFI snvoking oper—tions F F F F F F F F F F F F F F F F F F F PP PFRFP …pd—te dur—˜ility F F F F F F F F F F F F F F F F F F F F PP PFS ƒingleEo˜je™t re—d oper—tions F F F F F F F F F F F F F F F F F F PQ PFSFI ƒeri—liz—˜ility F F F F F F F F F F F F F F F F F F F F F F F PQ PFSFP ƒtri™t ƒeri—liz—˜ility F F F F F F F F F F F F F F F F F F F PQ W
  11. 11. 10 ¡ Quiver on the Edge: Consistent Scalable Edge Services PFT y˜je™t —v—il—˜ility in dyn—mi™ ™onditions F F F F F F F F F F F PS PFTFI his™onne™tions F F F F F F F F F F F F F F F F F F F F F F PS PFTFP ve—ves F F F F F F F F F F F F F F F F F F F F F F F F F F F PU PFU gorre™tness F F F F F F F F F F F F F F F F F F F F F F F F F F F F PW PFUFI €roof of seri—liz—˜ility F F F F F F F F F F F F F F F F F F QI PFUFP €roof of stri™t seri—liz—˜ility F F F F F F F F F F F F F F F QS PFV ynline ˜ookstore on the edge F F F F F F F F F F F F F F F F F F RQ PFVFI „€gE‡ overview F F F F F F F F F F F F F F F F F F F F F RQ PFVFP y˜je™t denitions F F F F F F F F F F F F F F F F F F F F RS PFVFQ fookstore inter—™tions F F F F F F F F F F F F F F F F F F RT PFW iv—lu—tion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F RV PFWFI ixperiment—l setup F F F F F F F F F F F F F F F F F F F F RW PFWFP f—seline tests F F F F F F F F F F F F F F F F F F F F F F F SI PFWFQ gomputeEintensive worklo—ds F F F F F F F F F F F F F F SR PFWFR ‡orklo—ds with oper—tion lo™—lity F F F F F F F F F F F SR PFWFS xetwork tr—™ ™l—ssi™—tion servi™e F F F F F F F F F F TPQ ‚oll˜—™k ett—™ks —nd hete™tion UI QFI ‚el—ted work F F F F F F F F F F F F F F F F F F F F F F F F F F F UP QFP ƒystem model extensions for roll˜—™k —tt—™ks F F F F F F F F F UR QFQ €roperties F F F F F F F F F F F F F F F F F F F F F F F F F F F F F UT QFQFI FORKS X ƒystemEwide fork ™onsisten™y F F F F F F F F F UT QFQFP FORKO X y˜je™tE˜—sed fork ™onsisten™y F F F F F F F F F UU QFR yverview of FORKO implement—tion F F F F F F F F F F F F F F UW QFS ster—tive h—shing ˜—sed en™oding F F F F F F F F F F F F F F F F VP QFSFI his™ussion F F F F F F F F F F F F F F F F F F F F F F F F F VR QFSFP ƒumm—ry F F F F F F F F F F F F F F F F F F F F F F F F F VS QFT ƒe™urity F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VS QFU yther ™onsider—tions F F F F F F F F F F F F F F F F F F F F F F F VV QFUFI heni—l of servi™e F F F F F F F F F F F F F F F F F F F F F VV QFUFP euthenti™—ted oper—tions F F F F F F F F F F F F F F F F VW QFV eppli™—tion to distri˜uted le systems F F F F F F F F F F F F F WH QFVFI ƒtor—ge ™osts F F F F F F F F F F F F F F F F F F F F F F F WI
  12. 12. Contents ¡ 11 QFVFP f—ndwidth —nd ™omput—tion ™osts F F F F F F F F F F F WR QFVFQ gost ™omp—rison with ƒ…xh‚ F F F F F F F F F F F F F WRR histri˜uted p—ultE„oler—nt „rees WU RFI ‚el—ted work F F F F F F F F F F F F F F F F F F F F F F F F F F F WV RFP f—™kground m—teri—l F F F F F F F F F F F F F F F F F F F F F F F WW RFPFI ‚—ndom regul—r gr—phs F F F F F F F F F F F F F F F F F WW RFPFP …niform s—mpling using r—ndom w—lks F F F F F F F F F IHH RFQ ƒystem model —nd go—ls F F F F F F F F F F F F F F F F F F F F F IHI RFR histri˜uted exp—nder ™onstru™tion F F F F F F F F F F F F F F F IHP RFRFI ‚—ndom —lmostEregul—r gr—phs F F F F F F F F F F F F F IHQ RFRFP fi—sed irreversi˜le r—ndom w—lks F F F F F F F F F F F F IHR RFRFQ ‚edu™ing mess—ge ™omplexity F F F F F F F F F F F F F F IHU RFRFR vo—d ˜—l—n™ing F F F F F F F F F F F F F F F F F F F F F F IHV RFRFS ƒumm—ry F F F F F F F F F F F F F F F F F F F F F F F F F IIH RFRFT €roof of exp—nsion F F F F F F F F F F F F F F F F F F F F IIP RFS „ree re™onstru™tion —fter f—ilures F F F F F F F F F F F F F F F F IIS RFT ƒimul—tion results F F F F F F F F F F F F F F F F F F F F F F F F IIUS histri˜uted ƒelfEyptimizing „rees IPQ SFI ‚el—ted work F F F F F F F F F F F F F F F F F F F F F F F F F F F IPQ SFP ƒystem model F F F F F F F F F F F F F F F F F F F F F F F F F F F IPS SFQ yverview F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPS SFR pl—ttening —lgorithms F F F F F F F F F F F F F F F F F F F F F F IPU SFRFI fottomEup —ttening F F F F F F F F F F F F F F F F F F F IPV SFRFP „opEdown semiE—ttening F F F F F F F F F F F F F F F F IQI SFRFQ ry˜rid —ttening F F F F F F F F F F F F F F F F F F F F F IQP SFRFR uE—ry trees F F F F F F F F F F F F F F F F F F F F F F F F IQR SFRFS €reserving geogr—phi™ lo™—lity F F F F F F F F F F F F F F IQS SFS ‚estru™turing ™ost —n—lysis F F F F F F F F F F F F F F F F F F F IQS SFT sntegr—tion with uiver9s ™onsisten™y proto™ols F F F F F F F F IRH SFU ixperiments F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRQ SFUFI ixperiment—l setup F F F F F F F F F F F F F F F F F F F F IRU
  13. 13. 12 ¡ Quiver on the Edge: Consistent Scalable Edge Services SFUFP ‚—ndom treeD r—ndom worklo—d F F F F F F F F F F F F F IRV SFUFQ ‚—ndom treeD group worklo—d F F F F F F F F F F F F F F IRV SFUFR qeogr—phi™ treeD r—ndom worklo—d F F F F F F F F F F F ISIT gon™lusions ISSfi˜liogr—phy ISU
  14. 14. Figures PFI histri˜uted queue m—inten—n™e F F F F F F F F F F F F F F F F F IT PFP histri˜uted queue implement—tion F F F F F F F F F F F F F F F IU PFQ €seudo™ode for o˜je™t lo™—tion —nd migr—tion F F F F F F F F F IW PFR ƒtri™tly seri—liz—˜le singleEo˜je™t re—d implement—tion F F F F PR PFS r—ndling proxy dis™onne™tions F F F F F F F F F F F F F F F F F PT PFT €seudo™ode for h—ndling proxy dis™onne™tions F F F F F F F F F PU PFU werging p—rent —nd ™hild9s queues F F F F F F F F F F F F F F F PV PFV €seudo™ode for — proxy9s promotion F F F F F F F F F F F F F F PW PFW wƒ„ of xorth emeri™—n €l—netv—˜ nodes F F F F F F F F F F F SI PFIH wi™ro˜en™hm—rk resultsX v—rying fr—™tion of re—ds F F F F F F F SS PFII wi™ro˜en™hm—rk resultsX v—rying num˜er of o˜je™ts F F F F F F ST PFIP wi™ro˜en™hm—rk resultsX multiEo˜je™t oper—tions F F F F F F F SU PFIQ wi™ro˜en™hm—rk resultsX v—rying num˜er of proxies F F F F F F SV PFIR wi™ro˜en™hm—rk resultsX dyn—mi™ tree ™onditions F F F F F F F SW PFIS gompute intensive worklo—d F F F F F F F F F F F F F F F F F F F TH PFIT y˜je™t popul—rity ˜i—s worklo—d F F F F F F F F F F F F F F F F TQ PFIU wƒ„ of €l—netv—˜ nodes in dierent ™ontinents F F F F F F F F TR PFIV ‚egion—l —™tivity worklo—d F F F F F F F F F F F F F F F F F F F F TS PFIW ghp of model sizes —nd upd—te times F F F F F F F F F F F F F TU PFPH fuilding tr—™ modelsX v—rying num˜er of proxies F F F F F F TW PFPI fuilding tr—™ modelsX multiEo˜je™t oper—tions F F F F F F F F UH QFI ‚e—™h—˜ility veri™—tion for fork ™onsisten™y F F F F F F F F F F VR QFP pork ™onsisten™y using ™ollision resist—nt h—sh fun™tion F F F F VU IQ
  15. 15. 14 ¡ Quiver on the Edge: Consistent Scalable Edge Services QFQ emount of sh—ring in distri˜uted le system us—ge F F F F F F WQ QFR myGap in distri˜uted le system us—ge F F F F F F F F F F F F F WQ RFI elgorithm to gener—te @d; AEregul—r r—ndom gr—ph F F F F F F IHR RFP …sing MDwalks with BIwalks to redu™e root lo—d F F F F F F F III RFQ ‚e™onne™ting tree p—rtitions using the exp—nder F F F F F F F F IIS RFR „ree m—inten—n™e pseudo™ode F F F F F F F F F F F F F F F F F F IIT RFS ixp—nsion —nd ™onne™tivityX ™onvergen™e F F F F F F F F F F F F IIW RFT ixp—nsion —nd ™onne™tivityX resilien™e to f—ilures F F F F F F F IPH RFU ixp—nsion —nd ™onne™tivityX pro™ess degrees F F F F F F F F F F IPH RFV vo—d on higher pro™esses in the tree F F F F F F F F F F F F F F IPP SFI fottomEup —ttening F F F F F F F F F F F F F F F F F F F F F F F IPV SFP €seudo™ode for fottomEup —ttening F F F F F F F F F F F F F F IPW SFQ „opEdown semiE—ttening F F F F F F F F F F F F F F F F F F F F IQP SFR €seudo™ode for topEdown semiE—ttening F F F F F F F F F F F F IQQ SFS €seudo™ode for hy˜rid —ttening F F F F F F F F F F F F F F F F IQQ SFT ry˜rid —ttening demonstr—tion F F F F F F F F F F F F F F F F IQR SFU localQ m—inten—n™e with —ttening F F F F F F F F F F F F F F F IRP SFV pl—ttening perform—n™eX r—ndom treeD r—ndom worklo—d F F F IRW SFW ix—mple topologiesX r—ndom treeD r—ndom worklo—d F F F F F IRW SFIH pl—ttening perform—n™eX r—ndom treeD group worklo—d F F F F ISH SFII ix—mple topologiesX r—ndom treeD group worklo—d F F F F F F F ISI SFIP pl—ttening perform—n™eX geogr—phi™ treeD r—ndom worklo—d F ISP SFIQ ix—mple topologiesX geogr—phi™ treeD r—ndom worklo—d F F F F ISQ
  16. 16. Tables QFI gost ™omp—rison with ƒ…xh‚ F F F F F F F F F F F F F F F F F WS SFI ‚oute ™h—nges —t t during ˜ottomEup —ttening F F F F F F F F IRQ SFP ‚oute ™h—nges —t z during ˜ottomEup —ttening F F F F F F F F IRR SFQ ‚oute ™h—nges —t x during ˜ottomEup —ttening F F F F F F F IRR SFR ‚oute ™h—nges —t a during ˜ottomEup —ttening F F F F F F F F IRS SFS ‚oute ™h—nges —t y during — topEdown semiE—ttening F F F F IRS SFT ‚oute ™h—nges —t x during — topEdown semiE—ttening F F F F IRT SFU ‚oute ™h—nges —t c during — topEdown semiE—ttening F F F F IRT SFV ‚oute ™h—nges —t a during — topEdown semiE—ttening F F F F IRT IS
  17. 17. 16 ¡ Quiver on the Edge: Consistent Scalable Edge Services
  18. 18. 1 Introductionhyn—mi™ we˜ servi™es —re ex—mples of snternetEs™—le —ppli™—tions th—t utiElize mut—˜le o˜je™tsF pollowing the su™™ess of ™ontent distri˜ution networks@ghxsA for st—ti™ ™ontentD numerous re™ent propos—ls —ttempt to s™—le dyEn—mi™ we˜ servi™es ˜y employing servi™e proxies —t the ’edge4 of the snterEnet @eFgFD see h—vis et —lF ‘PHHR“Y „—temur— et —lF ‘PHHQ“ —nd the referen™esthereinAF „his —ppro—™h h—s the potenti—l to ˜oth distri˜ute the oper—tionpro™essing lo—d —mong the proxiesD —nd to en—˜le ™lients to —™™ess the serEvi™e ˜y ™ommuni™—ting with ne—r˜y proxiesD r—ther th—n — potenti—lly dist—nt™entr—lized serverF e m—jor ™h—llenge in re—lizing this —r™hite™ture for dynamic we˜ servi™esDhoweverD is to en—˜le the @glo˜—lly distri˜utedA servi™e proxies to e™iently—™™ess the mut—˜le servi™e o˜je™ts for servi™ing ™lient oper—tionsD while enEsuring strong consistency sem—nti™s for these o˜je™t —™™essesF gonsistent o˜Eje™t sh—ring —mong the proxies en—˜les them to export the s—me ™onsistentview of the servi™e to the ™lientsD in turnF e™hieving even just seri—liz—˜ility@€—p—dimitriou ‘IWUW“Y fernstein et —lF ‘IWVU“A for oper—tions exe™uted —tthese proxies using st—nd—rd repli™—tion —ppro—™hes requires th—t — proxyinvolve either — ™entr—lized server or other @possi˜ly dist—ntA proxies on the™riti™—l p—th of e—™h upd—te oper—tionF ƒtri™t seri—liz—˜ility @€—p—dimitriou‘IWUW“A vi— su™h te™hniques requires wideE—re— inter—™tions for re—dsD —s wellF „his dissert—tion des™ri˜es — system ™—lled uiver th—t —ddresses this™h—llenge —nd —llows edge proxies to perform ™onsistent oper—tions on sh—redo˜je™tsD without overlo—ding some ™entr—lized ™oordin—tion pointD —nd withEout ™ont—™ting dist—nt proxies for e—™h oper—tionF ‡e —lso present extensions I
  19. 19. 2 ¡ Quiver on the Edge: Consistent Scalable Edge Servicesto uiver th—t provide — reli—˜le —nd selfEoptimizing ™ommuni™—tion networkfor uiver proxiesF sn this ™h—pterD we ˜riey motiv—te the design of theseproto™ols —nd summ—rize our ™ontri˜utionsF1.1 Consistent object access„his dissert—tion demonstr—tes —n —ltern—tive to the tr—dition—l repli™—tion˜—sed —ppro—™hes for —™hieving ™onsistent —™™ess to o˜je™ts ˜y edge proxEiesD while ret—ining the proxies9 lo—dEdispersing —nd l—ten™yEredu™ing ee™tsFuiver org—nizes the proxies in — tree rooted —t the serverY the tree is stru™Etured so th—t geogr—phi™—lly ™lose proxies reside ™lose to one —nother in thetreeF „o perform —n upd—te oper—tionD or —n oper—tion involving multipleo˜je™tsD — proxy uses the tree to migrate e—™h involved o˜je™t to itself —ndthen performs the oper—tion lo™—lly|thus seri—lizing these oper—tions —nd—™hieving the required ™onsisten™y sem—nti™sF „hough this —ppro—™h in™ursthe expense of o˜je™t migr—tion for upd—te —nd multiEo˜je™t oper—tions|—ndso is re—son—˜le only if o˜je™ts —re not too l—rge —nd oper—tions involve only— few|it —lso promises perform—n™e ˜enets for two types of —ppli™—tionsF „he rst type —re —ppli™—tions in whi™h oper—tions exhi˜it geogr—phi™lo™—lityX yn™e —n o˜je™t h—s ˜een migr—ted to — proxyD other oper—tions@in™luding upd—tesA —t th—t proxy involving this o˜je™t ™—n ˜e performedlo™—llyD in ™ontr—st to st—nd—rd repli™—tion te™hniquesF purthermoreD evenoper—tions —t ne—r˜y proxies ˜enetD sin™e the o˜je™t is —lre—dy ™lose —ndneed not ˜e migr—ted f—rY our use of — treeD through whi™h migr—tions o™E™urD is key to re—lizing this ˜enetF qiven the wellEknown diurn—l p—ttern of—ppli™—tion —™tivity th—t is syn™hronized with the ˜usiness d—yD —nd the f—™tth—t the ˜usiness d—y o™™upies dierent —˜solute times —round the worldDwe ˜elieve th—t exploiting worklo—d lo™—lity through migr—tion ™—n pl—y —nimport—nt role in optimizing glo˜—l —ppli™—tionsF „he se™ond type of —ppliE™—tions th—t ™—n ˜enet from the uiver p—r—digm —re those th—t involveeither l—rge —mounts of d—t— th—t would ˜e expensive to send to the server or™omputeEintensive oper—tions th—t would overlo—d the serverD sin™e uiver
  20. 20. 1.1 Consistent object access ¡ 3disperses the lo—d indu™ed ˜y ™lient oper—tions —™ross proxies r—ther th—n™entr—lizing it in the serverF €erh—ps the most o˜vious dr—w˜—™k of o˜je™t migr—tion is in™re—sedsensitivity to proxy dis™onne™tionsX sf — proxy dis™onne™ts while holding —no˜je™tD either ˜e™—use the proxy f—ilsD ˜e™—use it ™—n no longer ™ommuni™—tewith its p—rentD or ˜e™—use its p—rent dis™onne™tsD then oper—tions th—t itre™ently —pplied to the o˜je™t m—y ˜e lostF sn uiverD howeverD the ™onne™ted™omponent of the tree ™ont—ining the serverI ™—n e™iently regener—te thel—st version of the o˜je™t seen in th—t ™omponent when su™h — dis™onne™tionis dete™tedF „husD the server never loses ™ontrol of the servi™e o˜je™tsD —ndon™e —n o˜je™t re—™hes — portion of the tree th—t st—ys ™onne™ted @ex™ept forvolunt—ry dep—rturesAD —ll oper—tions it ree™ts ˜e™ome dur—˜leF por these dur—˜le oper—tionsD uiver ™—n implement either seri—liz—˜ility@€—p—dimitriou ‘IWUW“Y fernstein et —lF ‘IWVU“A or stri™t seri—liz—˜ility @€—Ep—dimitriou ‘IWUW“AF „he only dieren™e in the two modes is in how singleEo˜je™t re—ds —re h—ndledF sn neither ™—se do singleEo˜je™t re—ds require o˜je™tmigr—tionD —nd if merely seri—liz—˜ility su™esD then — proxy ™—n perform —singleEo˜je™t re—d lo™—llyF woreoverD re™—ll th—t stri™t seri—liz—˜ility impliesline—riz—˜ility @rerlihy —nd ‡ing ‘IWWH“A for —ppli™—tions th—t employ onlysingleEo˜je™t oper—tionsF gh—pter P det—ils these proto™ols —nd reports on —n ev—lu—tion ˜—sed onexperiments performed on €l—netv—˜ @ghun et —lF ‘PHHQ“A —nd — lo™—l isol—ted™lusterF „he €l—netv—˜ experiments me—sured the inherent ™osts of uiverthrough mi™ro˜en™hm—rks employing up to UH nodes in dierent ™ontinentsF‡e —lso ™omp—re uiver9s perform—n™e —g—inst — ™entr—lized implement—Etion —nd show the dr—sti™ improvement for worklo—ds th—t involve either™ompute intensive oper—tions or geogr—phi™ lo™—lity of referen™eF por the exEperiments performed on the lo™—l ™lusterD we implemented — network tr—™™l—ssi™—tion servi™e on top of uiver th—t ™omputes tr—™ ™l—ssiers fromdistri˜uted d—t— sour™esF gomputing these ™l—ssiers is — ™omput—tion—llyexpensive oper—tionD one th—t is not fe—si˜le to run on the resour™eEst—rved I We do not address the failure of the server; we presume it is rendered fault-tolerantusing standard techniques (e.g., Budhiraja et al. [1993]).
  21. 21. 4 ¡ Quiver on the Edge: Consistent Scalable Edge Services€l—netv—˜ nodesF ‡e —g—in ™omp—re the perform—n™e of this —ppli™—tion imEplemented using uiver —g—inst —n implement—tion ˜—sed on — ™entr—lizedserverF uiver outperforms the ™entr—lized servi™e ˜y orders of m—gnitudeunder v—rious worklo—ds for ˜oth upd—te —nd re—dEonly oper—tionsF1.2 Detecting misbehaving proxies„he proto™ol for ™onsistent o˜je™t —™™ess migr—tes o˜je™ts to proxies perEforming oper—tions on these o˜je™ts through other servi™e proxiesD —nd sothe ™orre™tness of this proto™ol depends on these intermediate proxies ˜eEh—ving —™™ording to their spe™i™—tionsF „his is — r—ther strong —ssumptionD™onsidering these servi™e proxies —re geogr—phi™—lly distri˜utedD often noteven under dire™t —dministr—tive ™ontrol of the entity oering the servi™eF ‡eD thereforeD developed —n extension to uiver th—t —llows the honestproxies to dete™t mis˜eh—ving intermedi—te proxies —ttempting to viol—te™onsisten™y sem—nti™s provided ˜y uiverP F sn p—rti™ul—rD when —n intermeEdi—te proxy is ™ompromisedD one —tt—™k it ™—n mount is — rollback attackD inwhi™h it suppresses some oper—tions from re—™hing other proxiesF fyz—ntinef—ultEtoler—nt repli™—tion of the intermedi—te proxies ™—n dete™t @eFgFD ƒhin—nd ‚—m—n—th—n ‘IWVU“Y elvisi et —lF ‘PHHI“Y fuskens —nd ‚F €F fi—n™hini‘IWWQ“A or m—sk @eFgFD v—mport ‘IWUV“Y ƒ™hneider ‘IWWH“Y ‚eiter —nd firm—n‘IWWR“Y g—stro —nd viskov ‘PHHP“Y g—™hin —nd €oritz ‘PHHP“Y ‰in et —lF ‘PHHQ“Ye˜dEilEw—lek et —lF ‘PHHS“A su™h mis˜eh—viorD ˜ut gener—lly introdu™es sigEni™—nt perform—n™eD m—n—gement —nd h—rdw—re ™ostsF sn our settingD su™h™osts —re un—™™ept—˜le —nd so the proxies —re for™ed to rely upon untrustedintermedi—riesF €revention of the roll˜—™k —tt—™kD thusD ˜e™omes impossi˜leD—nd the ˜est one ™—n hope for is dete™tion through fork consistency @w—zires e—nd ƒh—sh— ‘PHHP“Y vi et —lF ‘PHHR“AF sn — nutshellD fork ™onsisten™y ensures th—t if @the result ofA —n oper—tionop is o˜served ˜y two honest proxiesD then these proxies per™eive the s—mesequen™e of oper—tions to h—ve ˜een performed to th—t pointD iFeFD up to P Defending against other types of malicious behavior by proxies, e.g., corrupting serviceobjects or denial of service is not addressed in this thesis.
  22. 22. 1.2 Detecting misbehaving proxies ¡ 5—nd in™luding op F „he utility of this property is perh—ps more ™le—r whenit is st—ted in the ™ontr—positiveX if the ™ompromised intermedi—ries permit—n oper—tion to ˜e visi˜le to one proxy ˜ut suppress it from —notherD thensu˜sequently these two proxies will never see —ny oper—tion in ™ommonYthese proxies —re s—id to ˜e ’forked4F yutEofE˜—nd ™ommuni™—tion ˜etweenthese proxies then en—˜les them to dete™t th—t they —re forkedF por ex—mpleDone proxy ™—n —pply —n oper—tion —nd the other proxy tests if it ™—n viewthe ee™ts of this oper—tionF sf notD the proxies ™onrm th—t they —re forked—nd ™—n initi—te —™tion toD eFgFD identify the mis˜eh—ving intermedi—ryF „his thesis explores —n —ltern—tive formul—tion of fork ™onsisten™y th—toers qu—lit—tively simil—r properties —g—inst the roll˜—™k —tt—™kD with su˜Est—nti—lly lower overhe—dF smpli™it in the denition of fork ™onsisten™y isth—t when the intermedi—ries —re not mis˜eh—vingD oper—tions —re syn™hroEnized to yield — seri—l order of —ll oper—tionsD independent of the o˜je™ts theyinvolveF yur rel—x—tion of fork ™onsisten™y enfor™es — seri—l order on only opEer—tions on the s—me o˜je™tD —nd —s su™hD it permits oper—tions th—t involvedistin™t o˜je™ts to pro™eed with the full ™on™urren™y th—t would otherwise˜e —llowed ˜y uiver9s ™onsisten™y proto™olsF et the s—me timeD it rem—insth—t proxies whose views of —n o˜je™t —re forked ™—n e—sily dete™t if they —reDsimply ˜y seeing if one ™—n modify the o˜je™t in — w—y th—t the other ™—n seethe modi™—tionF @yn the other h—ndD proxies who —re unsuspe™tingly forked˜y —n intermedi—ry on one o˜je™t might ˜e permitted to inter—™t norm—llyvi— —nother o˜je™tD —nd so — fork might persist longer in our model withoutdete™tionFA sn —dditionD the ™ryptogr—phi™ me™h—nisms th—t we employ inour implement—tion —re su˜st—nti—lly simpler —nd less expensive th—n thoseutilized in previous implement—tionsF „he fork ™onsisten™y implement—tionof ƒ…xh‚ @vi et —lF ‘PHHR“Y w—zires —nd ƒh—sh— ‘PHHP“A|the semin—l —nd emost ™omprehensive tre—tment of fork ™onsisten™y to d—te|involves — digiEt—l sign—ture per oper—tionD —long with overhe—d per oper—tion th—t is line—rin the tot—l num˜er of p—rti™ip—ntsF sn ™ontr—stD our solution h—s neither ofthese —s ™h—r—™teristi™sD —nd in f—™t employs ™ollisionEresist—nt h—shing —s its™ommonE™—se ™ryptogr—phi™ oper—tionF
  23. 23. 6 ¡ Quiver on the Edge: Consistent Scalable Edge Services gh—pter Q provides the det—iled —lgorithms employed ˜y uiver to gu—rE—ntee fork ™onsisten™yF yur —ppro—™h is very generi™ in th—t it is not tiedto other me™h—nisms in uiver —nd is equ—lly —ppli™—˜le to other dom—insFsn p—rti™ul—rD we show th—t our formul—tion of fork ™onsisten™y permits theimplement—tion of —
  24. 24. le service th—t oers qu—lit—tively simil—r defense toroll˜—™k —s ƒ…xh‚ ˜ut with su˜st—nti—lly ˜etter perform—n™eF ‡e use —n—lEysis —nd tr—™eEdriven simul—tions to show th—t the ™osts per oper—tion —resigni™—ntly redu™ed ˜y our —ppro—™hF ‡e —ddition—lly des™ri˜e how our—ppro—™h ™—n ˜e integr—ted into other distri˜uted o˜je™t sh—ring proto™olsDsu™h —s peerEtoEpeer dire™tory —nd mutu—l ex™lusion proto™ols @hemmer —ndrerlihy ‘IWWV“Y ‚—ymond ‘IWVW“Y rel—ry et —lF ‘IWWR“Y x—imi et —lF ‘IWWT“AF1.3 Recovering from partitionsuiver employs — rooted tree —s the ™ommuni™—tion network ˜etween theservi™e proxiesF …sing — tree h—s sever—l —dv—nt—gesX pirst it —llows the deEsign of very simple proto™ols for lo™—ting —nd retrieving @together referred to—s migr—tingA servi™e o˜je™tsD —nd for seri—lizing these migr—tionsF ƒe™ondD—n overl—y tree th—t preserves the geogr—phi™ dist—n™e ˜etween proxies|iFeFD pl—™es geogr—phi™—lly ne—r˜y proxies ™lose to e—™h other in the tree|en—˜les the migr—tion proto™ols to exploit lo™—lity of referen™eF pin—llyD —rooted tree n—tur—lly denes — prim—ry p—rtition|the one ™ont—ining therootF „his —llows the o˜je™t m—n—gement proto™ols to dene proxies in theprim—ry p—rtition —s ’™onne™ted4|—nd so these must ˜e —˜le to —™™ess serEvi™e o˜je™ts|—nd proxies in other p—rtitions —s ’dis™onne™ted4|in whi™h™—se the o˜je™t —™™ess requests initi—ted —nd the oper—tions performed ˜ythese proxies ™—n ˜e ignored until they re™onne™tF „husD the server —lw—ysh—s ™ontrol over the servi™e o˜je™tsD even when some proxies dis™onne™tDp—rtitioning the networkF „he downside of using — tree stru™tureD howeverD is its vulner—˜ility toproxy —nd link f—iluresF sn p—rti™ul—rD — single proxy or link f—ilure ™—n p—rEtition the tree —nd m—ke the servi™e o˜je™ts un—v—il—˜le to the proxies th—tend up in p—rtitions not ™ont—ining the rootF ‡e —ddress this issue through
  25. 25. 1.3 Recovering from partitions ¡ 7—n extension th—t e™iently ˜uilds — logi™—l f—ultEtoler—nt ™ommuni™—tionnetwork overl—yed on top of uiver9s distri˜uted treeF „he overl—y network ™onstru™ted ˜y our distri˜uted —lgorithm is —n ex-panderF ixp—nders —re —n import—nt ™l—ss of gr—phs th—t h—ve found —ppliE™—tions in the ™onstru™tion of error ™orre™ting ™odes @eFgFD ƒipser —nd ƒpielEm—n ‘IWWT“AD deEr—ndomiz—tion @eFgFD ejt—i et —lF ‘IWVQ“AD —nd in the designof f—ultEtoler—nt swit™hing networks @eFgFD €ippenger —nd vin ‘IWWP“AF „hef—ult toler—n™e of exp—nders @qoerdt ‘IWWV“Y f—g™hi et —lF ‘PHHR“A is pre™iselywh—t motiv—ted their use in uiverF yur —lgorithm st—rts with proxies ™onEne™ted in — tree —nd pro™eeds to —dd edges to —™hieve —n exp—nderF ƒin™eexpli™it ™onstru™tions of exp—nders —re gener—lly very ™omplexD we present —™onstru™tion th—t ’—pproxim—tes4 — dEregul—r r—ndom gr—phD iFeFD — r—ndomgr—ph in whi™h every proxy h—s —lmost d neigh˜orsF e dEregul—r r—ndomgr—ph isD with —n overwhelming pro˜—˜ilityD — good exp—nder @priedm—n‘IWWI“AF ‡e prove th—t our —pproxim—tion —™hieves ™omp—r—˜le exp—nsionF „he ™ontri˜utions of this work rest prim—rily in three fe—turesF pirstDour —lgorithm is ™ompletely distri˜utedF „hough exp—nder gr—phs h—ve ˜eenstudied extensivelyD distri˜uted ™onstru™tion of exp—nder networks rem—ins— ™h—llenging pro˜lemF yur —lgorithms use only lo™—l inform—tion —t e—™hproxy th—t ™onsists of the identities of the proxy9s neigh˜ors in the treeF edire™t ™onsequen™e of this is s™—l—˜ility|our —lgorithm is ™—p—˜le of gener—tEing exp—nders e™iently even with — l—rge num˜er of proxiesF ‡e ˜ootstr—pthis —lgorithm using — novel te™hnique th—t —llows — proxy to s—mple otherproxies uniformly —t r—ndom from the tree with low mess—ge ™omplexityF ƒe™ondD our —lgorithm —d—pts to joinsD le—ves —nd f—ilures of proxiesF€revious —ttempts —t distri˜uted ™onstru™tion of r—ndom exp—nders @v—w—nd ƒiu ‘PHHQ“Y €—ndur—ng—n et —lF ‘PHHQ“Y qk—ntsidis et —lF ‘PHHR“A try to™onstru™t dEregul—r r—ndom gr—phs where every node @— proxyD in our settingAh—s ex—™tly d neigh˜orsF ƒu™h gr—phs —re di™ult to ™onstru™t —nd m—int—inin — dyn—mi™ distri˜uted settingY eFgFD most of these ™onstru™tions requirenodes to prop—g—te their st—te to other nodes ˜efore le—ving the networkF‡e follow — more pr—gm—ti™ —ppro—™hD in th—t we only require th—t proxiesh—ve ’™lose4 to d neigh˜orsF sn doing so we dene — new ™l—ss of r—ndom
  26. 26. 8 ¡ Quiver on the Edge: Consistent Scalable Edge Servicesgr—phs whi™h we ™—ll @d; AEregul—r r—ndom gr—phsF „hese gr—phs give usmore exi˜ility in de—ling with the dyn—mi™ n—ture of our networkD whilestill —™hieving f—ult toler—n™eF pin—llyD we present — novel distri˜uted —lgorithm th—t uses the overl—yexp—nder to keep the underlying tree ™onne™ted in the presen™e of f—ultsF „his—lgorithm works on — ’˜estEeort4 ˜—sis|in most ™—ses the —lgorithm is —˜leto su™™essfully p—t™h the tree when proxies f—ilD howeverD in the unlikely eventof — l—rge fr—™tion of proxies f—iling simult—neouslyD the —lgorithm might notsu™™eedF sn these ™—ses we require some of the proxies to reEjoin the treeusing the def—ult me™h—nismD eFgFD ˜y ™ont—™ting the rootF gh—pter R det—ils these —lgorithmsF ‡e report simul—tion results th—tshow the ee™tiveness of the overl—y exp—nder in toler—ting f—iluresD —nd the™ost of our —lgorithm in terms of its mess—ge ™omplexityF1.4 Restructuring for performanceƒin™e proxies lo™—te —nd retrieve o˜je™ts through the treeD the worst ™—seperform—n™e of these —lgorithms is proportion—l to the di—meter @longestp—th ˜etween two proxiesA of the treeF „he trivi—l solution to m—ke the tree’—t4 @every proxy is — ™hild of the serverA does not s™—le well|the server˜e™omes — ˜ottlene™kF „hereforeD the perform—n™e of oper—tions in uiverth—t involve migr—ting or ™opying o˜je™ts through the treeD ™—n ˜enet from— distri˜uted me™h—nism th—t would restru™ture the tree to redu™e its di—meEterD while keeping — low xed degree —nd while preserving the lo™—tionE—w—restru™ture in the treeF ‡eD thereforeD dis™uss — n—l extension to uiver th—t employs — noveldistri˜uted —lgorithmD ™—lled atteningD th—t improves the perform—n™e ofo˜je™t migr—tion —nd ™opying proto™olsF sn p—rti™ul—rD —ttening —™hievesthree propertiesF pirstD it ˜rings proxies frequently —™™essing the s—meo˜je™ts|—nd thus frequently —™™essing e—™h other for migr—tion or ™opyEing of o˜je™ts|™loser to e—™h other in the treeF ‡orklo—ds in sever—l —pEpli™—tions —re known to exhi˜it lo™—lityD in the sense th—t proxies th—t h—ve™ommuni™—ted in the p—st —re likely to ™ommuni™—te —g—in in the futureY
  27. 27. 1.4 Restructuring for performance ¡ 9su™h —ppli™—tions ™—n ˜enet gre—tly from —tteningF xote th—t in — degreeE™onstr—ined tree @eFgFD — kE—ry treeAD optimizing the —™™ess ˜etween — p—irof proxies @˜y ˜ringing them ™lose to one —notherAD ™ould ™oni™t with opEtimizing for —nother p—ir of proxiesF „his situ—tion is further ™ompli™—teddue to the distri˜uted n—ture of our —lgorithmX e—™h proxy is only —w—re ofits neigh˜ors in the treeD —nd h—s no inform—tion —˜out the rem—ining treetopologyF pl—ttening employs — distri˜uted —lgorithm th—t utilizes only lo™—linform—tion —t e—™h proxyD —nd nds — ˜—l—n™e —mong ™oni™ting optimiz—Etion de™isions ˜y restru™turing for — p—rti™ul—r p—ir of proxies while —t thes—me time preserving the ee™ts of re™ent restru™turing de™isions m—de forother p—irsF ƒe™ondD —ttening h—s — tenden™y to redu™e the di—meter of the treeDwithout ever expli™itly ˜—l—n™ing the treeF sn p—rti™ul—rD it redu™es the diE—meter of the ™omponent of the tree th—t sp—ns proxies involved in re™entoper—tionsF „hereforeD if the worklo—d shows no lo™—lity|eFgFD if e—™h proxy—™™esses — proxy ™hosen uniformly —t r—ndom from —ll proxies in the tree|then —ttening redu™es the di—meter of the whole treeD sin™e in this ™—se the™omponent ™ont—ining frequently —™™essed proxies would sp—n most of thetreeF pin—llyD the restru™turing steps —re —ll lo™—lD iFeFD e—™h restru™turing step—t — proxy involves either only dire™t neigh˜ors or —t most neigh˜ors ofneigh˜ors @proxies two hops —w—y from e—™h otherA in the treeF „his —llowssimple implement—tion of lo™—l poli™ies —t e—™h proxyD eFgFD — su˜tree ™onEt—ining proxies geogr—phi™—lly ™lose to e—™h other ™ould enfor™e — poli™y th—tprevents — geogr—phi™—lly dist—nt proxy from entering this su˜treeD —s thetree is restru™turedF purthermoreD this lo™—l restru™turing en—˜les proxiesto e—sily upd—te their routing inform—tion @used for o˜je™t migr—tionAD toree™t the new tree topologyF „he restru™turing —lgorithm is dis™ussed in gh—pter SF ‡e —n—lyti™—llyprove th—t —ttening in™urs — worstE™—se O@log nA —mortized ™ost per —tEtening oper—tionF ƒin™e the ™ost of this restru™turing is dire™tly tied to the™ost of —™™essing —nother proxy|restru™turing is performed —long the p—th˜etween the two proxies|the worst ™—se ™ost of proxy —™™esses ™losely folE
  28. 28. 10 ¡ Quiver on the Edge: Consistent Scalable Edge Serviceslows the O@log nA —mortized perform—n™e of restru™turingF ‡e —lso reportempiri™—l results from tests performed on €l—netv—˜ th—t v—lid—te this —n—lEysisF ‡e further implemented — oodE˜—sed —™™ess me™h—nism th—t runs on— treeD —nd —llows proxies to —™™ess other proxies in the treeF ‡e presentresults th—t demonstr—te the perform—n™e of this oodE˜—sed proto™ol usingour selfEoptimizing treeD —nd ™omp—re them to those o˜t—ined ˜y runningthe s—me proto™ol on — r—ndomly gener—ted st—ti™ tree over the s—me set ofproxiesF „he oodE˜—sed proto™ol shows signi™—nt perform—n™e g—ins whenutilizing the —ttening —lgorithmD —nd shows the gener—lity of our s™heme—nd its potenti—l —ppli™—tions to sever—l dierent proto™olsF1.5 Structure of this document„his thesis —ddresses issues th—t —re relev—nt to the deployment of ™onsisEtent —nd s™—l—˜le edge servi™esF roweverD the dierent pro˜lems —ddressedhere h—ve their origins in sever—l dierent dom—ins like ™onsisten™y protoE™olsD systems rese—r™hD d—t—˜—se theory —nd gr—ph theoryF „his diversityrequires —n independent tre—tment of the ˜—™kground —nd rel—ted rese—r™hfor e—™h of these pro˜lemsF sn —dditionD our solutions need to ˜e ev—lu—ted—g—inst other solutions developed in the p—rti™ul—r rese—r™h —re—F por e—™hissue —ddressedD we therefore tre—t the ™orresponding rel—ted work —nd theev—lu—tion independently —long with the des™ription of the spe™i™ pro˜lem—nd our proposed solutionF „he rem—inder of this thesis is org—nized —s followsX gh—pter P des™ri˜es—nd ev—lu—tes the proto™ols employed ˜y proxies for performing ™onsistentoper—tions on servi™e o˜je™tsD —nd for the —v—il—˜ility of servi™e o˜je™ts evenwhen some proxies dis™onne™tF gh—pter Q dis™usses the extensions th—t ™—n˜e used ˜y honest proxies to dete™t mis˜eh—ving intermedi—te proxies th—t—ttempt to viol—te ™onsisten™y of o˜je™t —™™essesD dis™usses the —ppli™—tion ofour —ppro—™h in settings other th—n uiverD —nd ™omp—res the perform—n™e™osts in™urred ˜y our —ppro—™h —g—inst existing solutionsF gh—pter R det—ilsour extensions th—t redu™e the downtime experien™ed ˜y dis™onne™ted proxEies ˜y ™onstru™ting — f—ultEtoler—nt network on top of the overl—y treeD th—t
  29. 29. 1.5 Structure of this document ¡ 11™—n ˜e used ˜y proxies for e™ient re™onne™tionF „he —lgorithm to heurisEti™—lly restru™ture the tree for perform—n™e g—ins in worklo—ds th—t exhi˜itlo™—lity is des™ri˜ed —nd ev—lu—ted in gh—pter SF ‡e n—lly ™on™lude ingh—pter TF
  30. 30. 12 ¡ Quiver on the Edge: Consistent Scalable Edge Services
  31. 31. 2 Consistent Object Sharing„his ™h—pter dis™usses the o˜je™t m—n—gement proto™ols th—t —llow uiverproxies to perform ™onsistent oper—tions on servi™e o˜je™tsD —™hieving eitherseri—liz—˜ility or stri™t seri—liz—˜ilityF ‡e —lso present proto™ols th—t ensure™ontinuous —™™ess to servi™e o˜je™ts ˜y ™onne™ted proxiesD even when otherproxies dis™onne™t or volunt—rily le—ve the servi™eF en extensive ev—lu—tionof these proto™ols is presented through experiments performed on €l—netv—˜—nd — lo™—l ™lusterF2.1 Related work€roviding ™onsistent —nd s™—l—˜le —™™ess to sh—red o˜je™ts is — topi™ with— ri™h rese—r™h historyF eppro—™hes of whi™h we —re —w—re th—t do not usemigr—tion ™—n ˜e pl—™ed on — spe™trumF yn one endD —ll upd—tes to —n o˜je™t—re performed —t one ’prim—ry4 lo™—tionY upd—tes or ™—™he inv—lid—tions —rethen pushed out to @re—dEonlyA ™—™hed ™opies @eFgFD vuo et —lF ‘PHHP“Y vi et —lF‘PHHQ“Y emiri et —lF ‘PHHQ“Y vi —nd hong ‘IWWR“Y €l—ttner —nd elonso ‘PHHR“Yylston et —lF ‘PHHS“Y ‚—˜inovi™h et —lF ‘PHHQ“AF yn the other endD o˜je™ts —rerepli™—ted —™ross — set of proxiesF „ypi™—lly —ny proxy ™—n servi™e upd—tesor re—dsD —nd proxies —re syn™hronized ˜y prop—g—ting upd—tes to —ll proxiesvi—D eFgFD group multi™—st @eFgFD emir et —lF ‘PHHP“A or epidemi™ @eFgFD rollid—yet —lF ‘PHHQ“A —lgorithmsY this —ppro—™h is often referred to —s the ’upd—te—nywhere4 —ppro—™hF fetween these extremes lie other solutionsF por exE—mpleD in the upd—teE—nywhere s™en—rioD syn™hronizing upd—tes with only —quorum of proxies @eFgFD q—o et —lF ‘PHHS“ employs quorums in the ™ontextof edge servi™esA redu™es the ™ommuni™—tion overhe—dF sn the prim—ryEsite IQ
  32. 32. 14 ¡ Quiver on the Edge: Consistent Scalable Edge Services—ppro—™hD using the prim—ry only to order oper—tions while pro™essing theoper—tions on other proxies redu™es lo—d on the prim—ry @eFgFD fernstein —nd qoodm—n ‘IWVI“Y yzsu —nd †—lduriez ‘IWWT“AF yur —ppro—™h dep—rts from these p—r—digms ˜y migr—ting o˜je™ts toproxies for use in upd—tesF es dis™ussed in ƒe™tion IFID this en—˜les proE™essing lo—d to ˜e ˜etter dispersed —™ross proxiesD in ™omp—rison to mostprim—ryEsite ˜—sed —ppro—™hesF st —lso provides ™ommuni™—tion s—vings in™omp—rison to —ll the —ppro—™hes —˜ove in ™ir™umst—n™es where upd—tesexhi˜it geogr—phi™ lo™—lityF „his is p—rti™ul—rly true if stri™t seri—liz—˜ilityis requiredD sin™e to implement this property with the —˜ove —ppro—™hesDwideE—re— ™rossings o™™ur on the ™riti™—l p—th of —ll oper—tionsF wigr—tion is — st—ple of distri˜uted ™omputingY work in this —re— sp—nsde™—desD eFgFD xutt—ll ‘IWWR“Y wiloji™i et —lF ‘PHHH“ oer useful surveysF w—ny ™previous studies in o˜je™t migr—tion h—ve dr—wn from motiv—tion simil—r tooursD n—mely ™oElo™—ting pro™essing —nd d—t— resour™esF roweverD to ourknowledgeD the —ppro—™hes in uiver for m—n—ging migr—tion —nd o˜je™tre—dsD —nd for re™overing from dis™onne™tionsD —re novelF „he only workof whi™h we —re —w—re th—t —pplies o˜je™t migr—tion to dyn—mi™ we˜ serEvi™es @ƒiv—su˜r—m—ni—n et —lF ‘PHHS“A does not h—ndle f—ilure of proxiesD supEports only singleEo˜je™t oper—tions —nd provides we—k ™onsisten™y sem—nti™sFuiver improves on —ll of these —spe™tsF yur —ppro—™h to migr—tion w—s most dire™tly inuen™ed ˜y distri˜utedmutu—l ex™lusion proto™olsD not—˜ly ‚—ymond ‘IWVW“Y x—imi et —lF ‘IWWT“Yhemmer —nd rerlihy ‘IWWV“F „hese proto™ols —llow nodes —rr—nged in — treeto retrieve sh—red o˜je™ts —nd perform oper—tions —tomi™—llyF ‡hile these—ppro—™hes —™hieve s™—l—˜ility —nd ™onsisten™yD they do not —ddress f—iluresFyur —ppro—™h —lso en—˜les ™onsistent multiEo˜je™t oper—tions —nd optimiz—Etions for singleEo˜je™t re—ds th—t —re not possi˜le in these prior —lgorithmsF2.2 System model and goalsyur system implements — servi™e with — design—ted server —nd —n un˜oundednum˜er of proxiesF ‡e generi™—lly refer to the server —nd the proxies —s
  33. 33. 2.2 System model and goals ¡ 15processesF „o support the servi™eD — proxy joins the servi™eY in doing soD it ispositioned within — tree rooted —t the serverF e proxy ™—n —lso volunt—rilyleave the servi™eF sf — pro™ess loses ™ont—™t with one of its ™hildrenD eFgFD due to the f—ilureof the ™hild or of the ™ommuni™—tion link to the ™hildD then the ™hild —nd —llother proxies in the su˜tree rooted —t the ™hild —re s—id to disconnectF „osimplify dis™ussionD we tre—t the dis™onne™tion of — proxy —s perm—nentD ormore spe™i™—llyD — dis™onne™ted proxy m—y reEjoin the servi™e ˜ut with — reEiniti—lized st—teF sn —n exe™utionD — proxy th—t joins ˜ut does not dis™onne™t@though it might le—ve volunt—rilyA is ™—lled connectedF „he servi™e en—˜les proxies @on ˜eh—lf of ™lientsA to invoke operations onobjectsF „hese oper—tions m—y ˜e reads or updatesF …pd—tes ™ompute objectinstances from other o˜je™t inst—n™esF en o˜je™t inst—n™e o is —n immut—˜lestru™ture with sever—l eldsD in™luding —n identi
  34. 34. er eld o:id —nd — versioneld o:verF ‡e refer to o˜je™t inst—n™es with the s—me identier —s versionsof the s—me o˜je™tF eny oper—tion th—t produ™es —n o˜je™t inst—n™e o —soutput t—kes —s input the previous versionD iFeFD —n inst—n™e oH su™h th—toH :id a o:id —nd oH :ver C I a o:verF yur system —pplies oper—tions ™onsistentlyX for —ny system exe™utionDthere is — set of oper—tions Durable th—t in™ludes —ll oper—tions performed˜y ™onne™ted pro™esses @—nd possi˜ly some ˜y proxies th—t dis™onne™tAD su™hth—t the ™onne™ted pro™esses per™eive the oper—tions in Durable @—nd no othEersA to ˜e exe™uted sequenti—llyF wore pre™iselyD we present two v—ri—tions ofour —lgorithmF yne enfor™es serializability @€—p—dimitriou ‘IWUW“Y fernsteinet —lF ‘IWVU“AX —ll ™onne™ted pro™esses per™eive the oper—tions in Durableto ˜e exe™uted in the s—me sequenti—l orderF „he other enfor™es —n evenstronger propertyD strict serializability @€—p—dimitriou ‘IWUW“AX the s—me seEquenti—l order per™eived ˜y pro™esses preserves the re—lEtime order ˜etweenoper—tionsF
  35. 35. 16 ¡ Quiver on the Edge: Consistent Scalable Edge Services2.3 Object management‡e ˜egin ˜y des™ri˜ing — highElevel —˜str—™tion in ƒe™tion PFQFI th—t en—˜lesour solutionD —nd then dis™uss the implement—tion of th—t —˜str—™tion inƒe™tion PFQFPF ƒe™tions PFR —nd PFS des™ri˜e how this implement—tion en—˜lesuiver proxies to perform servi™e oper—tionsFpigure PFIF @—A distQ with pro™esses aD bD c —nd dF @˜A e —ppends itself to distQ˜y sending — retrieve request to dF @™A ‡hen a ™ompletes its oper—tionD itmigr—tes the o˜je™t to b —nd drops o distQF2.3.1 distQ abstractionpor e—™h o˜je™tD pro™esses who wish to perform oper—tions on th—t o˜je™t—rr—nge themselves in — logi™—l distri˜uted pspy queue denoted distQD —ndt—ke turns —™™ording to their positions in distQ to perform those oper—tionsF„he pro™ess —t the front of distQ is denoted —s the head —nd the one —tthe end of distQ is denoted —s the tailF sniti—llyD distQ ™onsists of only onepro™ess|the serverF ‡hen —n oper—tion is invoked —t — pro™ess pD p sends— retrieve request to the ™urrent t—il of distQF „his request results in —ddingp to the end of distQD m—king it the new t—ilY see pigure PFIE@˜AF ‡hen thehe—d of distQ ™ompletes its oper—tionD it drops o the queue —nd migratesthe o˜je™t to the next pro™ess in distQD whi™h ˜e™omes the new he—dY seepigure PFIE@™AF „his distri˜uted queue ensures th—t the o˜je™t is —™™essedsequenti—llyF
  36. 36. 2.3 Object management ¡ 17 e pro™ess performs —n oper—tion involving multiple o˜je™ts ˜y migr—tinge—™h involved o˜je™t vi— its distQ to itselfF yn™e the pro™ess holds theseo˜je™tsD it performs its oper—tion —nd then rele—ses e—™h su™h o˜je™t to ˜emigr—ted to the pro™ess next in th—t o˜je™t9s distQFpigure PFPF ƒqu—res —t — pro™ess represent its localQY leftEmost squ—re is thehe—dF sniti—lly a h—s the o˜je™tF e requests from aD f requests from eD —nd amigr—tes the o˜je™t to eF2.3.2 distQ implementation„he ™ore of our —lgorithm implements distQ per o˜je™tF distQ for the o˜je™twith identier id @hen™eforthD distQ‘id “A is implemented using — lo™—l pspyqueue p:localQ‘id “ —t every pro™ess pF ilements of p:localQ‘id “ —re neigh˜orsof p in the treeF sntuitivelyD p:localQ‘id “ is m—int—ined so th—t the he—d —ndt—il of p:localQ‘id “ point to p9s neigh˜ors th—t —re in the dire™tion of thehe—d —nd t—il of distQ‘id “D respe™tivelyF sniti—llyD the server h—s the o˜je™t—nd it is the only element in distQ‘id “F „husD p:localQ‘id “ —t e—™h proxy p isiniti—lized with — single entryD p9s p—rentD the p—rent ˜eing in the dire™tionof the server @pigure PFPE@—AAF ‡hen — pro™ess p re™eives — retrieve request for the o˜je™t with identierid from its neigh˜or q D it forw—rds the request to the t—il of p:localQ‘id “ —nd—dds q to the end of p:localQ‘id “ —s the new t—ilF „husD the t—il of p:localQ‘id “now points in the dire™tion of the new t—il of distQ‘id “D whi™h must ˜e in thedire™tion of q sin™e the l—test retrieve request ™—me from qY see pigures PFPE@˜A —nd PFPE@™AF ‡hen — pro™ess p re™eives — migr—te mess—ge ™ont—ining the
  37. 37. 18 ¡ Quiver on the Edge: Consistent Scalable Edge Serviceso˜je™tD it removes the ™urrent he—d of p:localQ‘id “ —nd forw—rds the o˜je™tto the new he—d of p:localQ‘id “F „his ensures th—t the he—d of p:localQ‘id “points in the dire™tion of the new he—d of distQ‘id “D see pigure PFPE@dAF €seudo™ode for this —lgorithm is shown in pigure PFQF ‡e usethe following not—tion throughout for —™™essing localQX localQFhead —ndlocalQFtail —re the he—d —nd the t—ilF localQ:elmt‘i“ is the ith element@localQ:elmt‘I“ a localQ:headAF localQFsize is the ™urrent num˜er of elementsFlocalQFremoveFromHead@A removes the ™urrent he—dF localQ:addToTail@eA—dds the element e to the t—ilF localQFhasElements@A returns true if localQis not emptyF sniti—liz—tion of — pro™ess upon joining the tree is not shown inthe pseudo™ode of pigure PFQY we des™ri˜e initi—liz—tion hereF ‡hen — pro™essp joins the treeD it is initi—lized with — p—rent p:parent @c if p is the serverAFi—™h pro™ess —lso m—int—ins — set p:children th—t is initi—lly empty ˜ut th—tgrows —s other proxies —re —dded to the treeF por e—™h o˜je™t identier id Dp initi—lizes — lo™—l queue p:localQ‘id “ ˜y enqueuing p if p is the server —ndp:parent otherwiseF sn —dditionD for e—™h o˜je™t identier id D the server piniti—lizes its ™opy of the o˜je™tD p:objs‘id “D to — def—ult initi—l st—teF i—™h pro™ess ™onsists of sever—l thre—ds running ™on™urrentlyF „he glo˜—lst—te —t — pro™ess p th—t is visi˜le to —ll thre—ds is denoted using the ’p:4prexD eFgFD p:parentF †—ri—˜le n—mes without the ’p:4 prex represent st—telo™—l to its thre—dF sn order to syn™hronize these thre—dsD the pseudo™ode ofpro™ess p employs — sem—phoreI p:sem‘id “ per o˜je™t identier id D used toprevent the migr—tion of o˜je™t p:objs‘id “ to —nother pro™ess ˜efore p is doneusing itF p:sem‘id “ is initi—lized to one —t the server —nd zero elsewhereF yurpseudo™ode —ssumes th—t —ny thre—d exe™utes in isol—tion until it ™ompletesor ˜lo™ks on — sem—phoreF2.3.3 Migrating one object„he routing of retrieve requests for o˜je™ts is h—ndled ˜y thedoRetrieveRequest fun™tion shown in pigure PFQF ‡hen p exe™utes I To remind the reader, a semaphore s represents a non-negative integer with two atomicoperations: V (s) increments s by one; P (s) blocks the calling thread while s = 0 and thendecrements s by one.
  38. 38. 2.3 Object management ¡ 19doRetrieveRequest@from ; id ; prog A GB snvoked lo™—lly on request ˜y from BGIF hq; prog i 2 p:localQ‘id “:tail 0 GB q m—de the l—st request for this o˜je™t BGPF p:localQ‘id “:addToTail@hfrom ; prog iA GB xext request will ˜e forw—rded to from BGQF if q a p GB sf s l—st requested this o˜je™t FFF BGRF P @p:sem‘id “A GB FFFthen w—it till s —m done using it BGSF doMigrate@id A GB FFF—nd then migr—te to requesting pro™ess BGTF else GB sf s did not l—st request this o˜je™t FFF BGUF send @retrieveRequest X p; id A to q GB FFFthen forw—rd to who l—st requested it BGdoMigrate@id A GB snvoked lo™—lly for h—ndling migr—tion BGVF p:localQ‘id “:removeFromHead@A GB ywner not tow—rds ™urrent he—d now BGWF hq; prog i 2 p:localQ‘id “:head GB feing migr—ted tow—rds qD the new he—d BGIHF if q a p GB sf s requested this o˜je™t FFF BGIIF prog GB FFFthen exe™ute progr—m registered e—rlier BGIPF else if q a p:parent GB sf p—rent requested this o˜je™t FFF BG p:DepsIQF IDs 2 fid X id aA id g 0 0 GB FFFthen nd o˜je™ts this one depends on BGIRF Objs 2 fp:objs‘id 0 “ X id 0 P IDs g GB FFF™olle™t —ll these o˜je™ts BGISF DepSet 2 p:Deps ’ @IDs ¢ IDs A GB FFF—nd their dependen™y rel—tions BGITF send @migrate X p:objs‘id “; Objs ; DepSet A to q GB FFFsend everything to p—rent BGIUF p:Deps 2 p:Deps n DepSet GB FFFremove dependen™ies for future BGIVF else GB sf — ™hild requested this o˜je™tFFF BGIWF send @migrate X p:objs‘id “; Y; YA to q GB FFFthen just migr—te this o˜je™t BG…pon re™eiving @retrieveRequest X from ; id A GB ‚equest for id re™eived from from Ta p BGPHF doRetrieveRequest@from ; id ; cA GB snvoke doRetrieveRequest on from 9s ˜eh—lf BG…pon re™eiving @migrate X o; Objs ; DepSet A GB o is migr—ted —nd depends on Objs BGPIF p:objs‘o:id“ 2 o GB ƒ—ve the migr—ted o˜je™t o BGPPF foreach o0 P Objs GB por e—™h copied o˜je™t FFF BGPQF p:objs‘o0 :id“ 2 o0 GB FFF s—ve the ™opied o˜je™t BGPRF p:Deps 2 p:Deps ‘ DepSet GB …pd—te the lo™—l dependen™y rel—tion BGPSF doMigrate@o:idA GB snvoke doMigrate for id BG pigure PFQF y˜je™t m—n—gement pseudo™ode for pro™ess pdoRetrieveRequest@from ; id ; prog AD it —dds hfrom ; prog i to the t—il ofp:localQ‘id “ @line PAD sin™e from denotes the pro™ess from whi™h p re™eivedthe request for id F @prog h—s ˜een elided from dis™ussion of localQ so f—rY itwill ˜e dis™ussed in ƒe™tion PFRFA p then ™he™ks if the previous t—il @lines ID QAw—s itselfF sf soD it —w—its the ™ompletion of its previous oper—tion @line RA˜efore it migr—tes the o˜je™t to from ˜y invoking doMigrate@id A @line SDdis™ussed ˜elowAF sf the previous t—il w—s —nother pro™ess qD then p sends@retrieveRequest X p; id A to q @line UAY when re™eived —t qD q will performdoRetrieveRequest@p; id ; cA simil—rly @line PHAF sn this w—yD — retrieve request
  39. 39. 20 ¡ Quiver on the Edge: Consistent Scalable Edge Servicesis routed to the t—il of distQ‘id “D where it is ˜lo™ked until the o˜je™t migr—Etion ˜eginsF xote th—t p invokes doRetrieveRequest not only when it re™eives— retrieve request from —nother pro™ess @line PHAD ˜ut —lso to migr—te theo˜je™t for itselfF wigr—ting —n o˜je™t with identier id is h—ndled ˜y the doMigrate fun™EtionF ƒin™e the he—d of p:localQ‘id “ should point tow—rd the ™urrent lo™—Etion of the o˜je™tD p must remove its nowEst—le he—d @line VAD —nd identifythe new he—d q to whi™h it should migr—te the o˜je™t to re—™h its futuredestin—tion @line WAF sf th—t future destin—tion is p itselfD then p runs theprogr—m prog @line IIA th—t w—s stored when p requested the o˜je™t ˜y inEvoking doRetrieveRequest@p; id ; prog AY —g—inD we defer dis™ussion of prog toƒe™tion PFRF ytherwiseD p migr—tes the o˜je™t tow—rd th—t destin—tion @line ITor IWAF sf p is migr—ting the o˜je™t to — ™hild @line IWAD then it need not send—ny further inform—tionF sf p is migr—ting the o˜je™t to its p—rentD howeverDthen it must send —ddition—l inform—tion @lines IQ{ITA th—t is det—iled inƒe™tion PFQFRF2.3.4 Object dependencies„here is — n—tur—l dependen™y rel—tion A @pronoun™ed ’depends on4A ˜eE optween o˜je™t inst—n™esF pirstD dene o A oH if in —n oper—tion op D either opprodu™ed o —nd took oH —s inputD or o —nd oH were ˜oth produ™ed ˜y op F „henD S oplet A a op A F sntuitivelyD — proxy p should p—ss —n o˜je™t inst—n™e o top:parent only if —ll o˜je™t inst—n™es on whi™h o depends —re —lre—dy re™orded—t p:parentF ytherwiseD p:parent might re™eive only o ˜efore p dis™onne™tsDin whi™h ™—se —tomi™ity of the oper—tion th—t produ™ed o ™—nnot ˜e gu—r—nEteedF „husD to p—ss o to p:parentD p must copy —long —ll o˜je™t inst—n™es onwhi™h o dependsF xote th—t ™opying h—s dierent sem—nti™s th—n migr—tingD—nd in p—rti™ul—r ™opying does not tr—nsfer ’ownership4 of the o˜je™tF fe™—use e—™h pro™ess holds only the l—test version it h—s re™eived fore—™h o˜je™t identierD howeverD it m—y not ˜e possi˜le for p to ™opy —no˜je™t inst—n™e oH upw—rd when migr—ting o even if o A oH X oH m—y h—ve˜een ’overwritten4 —t pD iFeFD p:objs‘oH :id“:ver oH :verF sn this ™—seD it would
  40. 40. 2.4 Update and multi-object operations ¡ 21su™e to ™opy p:objs‘oH :id“ in lieu of oH D provided th—t e—™h oHH su™h th—tp:objs‘oH :id“ A oHH were —lso ™opied|˜ut of ™ourseD oHH might h—ve ˜een ’overEwritten4 —t pD —s wellF es su™hD in — renement of the initi—l —lgorithm —˜oveDwhen p migr—tes o to its p—rentD it ™omputes —n identier set IDs re™ursively—™™ording to the following rules until no more identiers ™—n ˜e —dded toIDs X @iA initi—lize IDs to fo:idgY @iiA if id P IDs —nd p:objs‘id “ A oH D then —ddoH :id to IDs F p then ™opies fp:objs‘id “gid PIDs to its p—rentF st is not ne™ess—ry for e—™h pro™ess p to tr—™k A ˜etween —ll o˜je™tinst—n™es in order to ™ompute the —ppropri—te identier set IDs F ‚—therDe—™h pro™ess m—int—ins — ˜in—ry rel—tion p:Deps ˜etween o˜je™t identiEersD initi—lized to YF sf p performs —n upd—te oper—tion op su™h th—t —n opoutput p:objs‘id “ A p:objs‘id H “D then p —dds @id ; id H A to p:DepsF sn order toperform doMigrate@id A to p:parentD p determines the identier set IDs —sthose indi™es re—™h—˜le from id ˜y following edges @rel—tionsA in p:Deps|re—™h—˜ility is denoted p:Deps in line IQ of pigure PFQ|—nd ™opies ˜oth aA H “gObjs a fp:objs‘id id PIDs @line IRA —nd DepSet a p:Deps ’ @IDs ¢ IDs A 0@line ISA —long with the migr—ting o˜je™t @line ITAF pin—llyD p upd—tesp:Deps 2 p:Deps n DepSet @line IUAD iFeFD to remove these dependen™ies forfuture migr—tions upw—rdF …pon re™eiving — migr—tion from — ™hild with ™opied o˜je™ts Objs —nddependen™ies DepSet D p s—ves Objs in p:objs @lines PP{PQA —nd —dds DepSetto p:Deps @line PRAF xote th—t the server @root of the treeA need not m—int—in—ny dependen™iesD sin™e it —lw—ys migr—tes or ™opies the o˜je™ts downw—rdsin the treeF2.4 Update and multi-object operationssn order to —™hieve our desired ™onsisten™y sem—nti™sD for e—™h o˜je™t weenfor™e sequenti—l exe™ution of —ll upd—te —nd multiEo˜je™t oper—tions th—tinvolve th—t o˜je™tF portun—telyD for m—ny re—listi™ worklo—dsD these typesof oper—tions —re —lso the le—st frequentD —nd so the ™ost of exe™uting themsequenti—lly need not ˜e prohi˜itiveF sn —dditionD this sequenti—l exe™ution
  41. 41. 22 ¡ Quiver on the Edge: Consistent Scalable Edge Servicesof upd—te —nd multiEo˜je™t oper—tions en—˜les signi™—nt optimiz—tions forsingleEo˜je™t re—ds @ƒe™tion PFSA th—t domin—te m—ny worklo—dsF2.4.1 Invoking operationsvet id I ; : : : ; id k denote distin™t identiers of the o˜je™ts involved @re—dor upd—tedA in —n upd—te or multiEo˜je™t oper—tion op F „o perform op Dpro™ess p re™ursively ™onstru™ts|˜ut does not yet exe™ute|— sequen™eprog H ; prog I ; : : : ; prog k of progr—ms —s followsD where ’k4 delimits — progr—mXprog H 2 k op Y NewDeps 2 f @id ; id HA X op p:objs‘id “ A p:objs‘id H “gY p:Deps 2 p:Deps ‘ NewDeps Y V @p:sem‘id I “AY : : : Y V @p:sem‘id k “A kprog i 2 k doRetrieveRequest@p; id i; prog i IA k€ro™ess p then exe™utes prog k F xote th—t prog k requests id k —ndD on™e th—tis migr—tedD prog k I is exe™uted @—t line II of pigure PFQAF „hisD in turnDrequests id k I D —nd so forthF yn™e id I h—s ˜een migr—tedD prog H is exe™utedF„his performs op —nd then upd—tes the dependen™y rel—tion p:Deps @seeƒe™tion PFQFRA with the new dependen™ies introdu™ed ˜y op F pin—llyD prog Hexe™utes — V oper—tion on the sem—phore for e—™h o˜je™tD permitting it tomigr—teF †iewing the sem—phores p:sem‘id I “D : : :D p:sem‘id k “ —s lo™ksD prog k™—n ˜e viewed —s implementing stri™t twoEph—se lo™king @fernstein et —lF‘IWVU“AF ƒoD to prevent de—dlo™kD id I ; : : : ; id k must ˜e —rr—nged in — ™—noni™—lorderF2.4.2 Update durabilitye proxy th—t performs —n upd—te oper—tion ™—n for™e the oper—tion to ˜edur—˜leD ˜y ™opying e—™h resulting o˜je™t inst—n™e o @—nd those on whi™hit dependsD see ƒe™tion PFQFRA to the serverD —llowing e—™h pro™ess p on thep—th to s—ve o if p:objs‘o:id“:ver o:verF „h—t s—idD doing so per upd—te wouldimpose — signi™—nt lo—d on the systemD —nd so our go—ls @ƒe™tion PFPA do not
  42. 42. 2.5 Single-object read operations ¡ 23require thisF ‚—therD our go—ls require only th—t — proxy for™es its upd—testo ˜e dur—˜le when it le—ves the tree @ƒe™tion PFTFPAD so th—t oper—tions˜y — proxy th—t rem—ins ™onne™ted until it le—ves —re dur—˜leF yper—tionsperformed —t the server —re dur—˜le ˜e™—use our model —ssumes th—t theserver never f—ilsF2.5 Single-object read operations‡e present two proto™ols implementing — singleEo˜je™t re—dF hepending onwhi™h of these two proto™ols is employedD our system gu—r—ntees either seriE—liz—˜ility or stri™t seri—liz—˜ility when ™om˜ined with the implement—tion ofupd—te —nd multiEo˜je™t oper—tions from ƒe™tion PFRF ‡e provide ™orre™tness—rguments for ˜oth versions of our proto™ols in ƒe™tion PFUF2.5.1 Serializabilityhue to the seri—l exe™ution of upd—te —nd multiEo˜je™t oper—tions @ƒe™Etion PFRAD singleEo˜je™t re—ds so —s to —™hieve seri—liz—˜ility @fernstein et —lF‘IWVU“A ™—n ˜e implemented with lo™—l re—ds|iFeFD — pro™ess p performs —re—d involving — single o˜je™t with identier id ˜y simply returning p:objs‘id “F2.5.2 Strict Serializability‚e™—ll th—t —ll upd—te —nd multiEo˜je™t re—d oper—tions involving the s—meo˜je™t —re performed seri—lly @ƒe™tion PFRAF „hereforeD in order to gu—r—nteestri™t seri—liz—˜ilityD it su™es th—t — singleEo˜je™t re—d oper—tion op on —no˜je™t with identier id invoked ˜y — pro™ess pD re—ds the l—test version ofthis o˜je™t produ™ed ˜efore op is invokedF „his ™ould ˜e —™hieved ˜y seri—lEizing op with the upd—te —nd multiEo˜je™t oper—tions in distQ‘id “F roweverDthis would require op to w—it for the ™ompletion of the ™on™urrent upd—te—nd multiEo˜je™t oper—tions @those performed ˜y pro™esses pre™eding p indistQ‘id “AF e more e™ient solution is to request the l—test version from the pro™ess—t the he—d of distQ‘id “|the pro™ess th—t is the ™urrent ’owner4 of the o˜je™t
  43. 43. 24 ¡ Quiver on the Edge: Consistent Scalable Edge Serviceswith identier id F yur —lgorithms —lre—dy provide — w—y to route to the he—dof distQ‘id “D using localQ‘id “Fhead —t e—™h pro™essF „hus — re—d request for idfollows p:localQ‘id “:head —t e—™h pro™ess p until it re—™hes — pro™ess pH su™hth—t either pH :localQ‘id “:head a pH @iFeFD pH holds the l—test o˜je™t versionAD orpH :localQ‘id “:head a pHH is the pro™ess th—t forw—rded this re—d request to pH Fsn the l—tter ™—seD pH forw—rded pH :objs‘id “ to pHH in — migr—tion ™on™urrentlywith pHH forw—rding this re—d request to pH @sin™e pHH :localQ‘id “:head a pH whenpHH did soAD —nd so it is s—fe for pH to serve the re—d request with pH :objs‘id “F „he initi—tor p of the re—d request ™ould en™ode its identity within therequestD —llowing the responder pH to dire™tly send — ™opy of the o˜je™t top outside the treeF roweverD to f—™ilit—te re™onstituting the o˜je™t in ™—se itis lost due to — dis™onne™tion @— me™h—nism dis™ussed in ƒe™tion PFTFIAD werequire th—t the o˜je™t ˜e p—ssed through the tree to the highest pro™ess inthe p—th from pH to pD iFeFD the lowest ™ommon —n™estor pHH of the initi—tor —ndresponder of the re—d requestF efter re™eiving the o˜je™t in response to there—d requestD pHH dire™tly sends the o˜je™t to p @the initi—torA outside the treeFxote th—t sin™e the requested o˜je™t is ™opied upw—rds in the tree from pHto pHH @unless pH a pHH AD —ny o˜je™ts th—t the requested o˜je™t depends uponDmust —lso ˜e ™opied —long using the te™hniques des™ri˜ed in ƒe™tion PFQFRFpigure PFRF p initi—tes — single o˜je™t re—d request th—t re—™hes pH F pH sendsthe response through the tree to the highest pro™ess pHH in the p—thF pHH then™opies the requested o˜je™t dire™tly to p outside the treeF
  44. 44. 2.6 Object availability in dynamic conditions ¡ 252.6 Object availability in dynamic conditionsyur —lgorithms m—ke no —ssumptions —˜out how proxies join the treeD —ndthis me™h—nism ™—n ˜e t—ilored to —ppli™—tion needs|eFgFD in our experiEments @ƒe™tion PFWA we ™onstru™t — minimum sp—nning tree of proxies ˜—sedon network l—ten™iesF rere we det—il how to —d—pt our —lgorithm to —ddressdis™onne™tions @ƒe™tion PFTFIA —nd proxies le—ving volunt—rily @ƒe™tion PFTFPAF2.6.1 Disconnections‚e™—ll th—t when — pro™ess loses ™ont—™t with — ™hildD —ll proxies in the su˜Etree rooted —t th—t ™hild —re s—id to disconnectF „he ™hild @orD if the ™hildf—iledD e—™h uppermost surviving proxy in the su˜treeAD informs its su˜treeof the dis™onne™tionD to en—˜le proxies to re™onne™t @—fter reiniti—lizingA ifdesiredF yf ™on™ern hereD howeverD is th—t some of these dis™onne™ted proxEies m—y h—ve e—rlier issued retrieve requests for o˜je™tsD —nd for e—™h su™ho˜je™t with identier id D the dis™onne™ted proxy m—y —ppe—r in distQ‘id “Fsn this ™—seD it must ˜e ensured th—t the ™onne™ted pro™esses pre™eded ˜y —dis™onne™ted proxy in distQ‘id “ ™ontinue to m—ke progressF „o this endD —llo™™urren™es of the dis™onne™ted proxies in distQ‘id “ —re repl—™ed with thep—rent p of the uppermost dis™onne™ted proxy qD see pigure PFSF ghoosing p to repl—™e the dis™onne™ted proxies is motiv—ted ˜y sever—lf—™torsX pirstD p is in the ˜est position to dete™t the dis™onne™tion of thesu˜tree rooted —t its ™hild qF ƒe™ondD —s we will see ˜elowD in our —lgorithmp need only t—ke lo™—l —™tions to repl—™e the dis™onne™ted proxiesY —s su™hDthis is — very e™ient solutionF „hirdD in ™—se the he—d of distQ‘id “ is oneof the dis™onne™ted proxiesD the o˜je™t with identier id must ˜e in the disE™onne™ted ™omponentF „his o˜je™t needs to ˜e re™onstituted using the lo™—l™opy —t one of the pro™esses still ™onne™tedD while minimizing the num˜er ofupd—tes ˜y nowEdis™onne™ted proxies th—t —re lostF p is the ˜est ™—ndid—te—mong the stillE™onne™ted pro™essesX p is the l—st to h—ve s—ved the o˜je™t—s it w—s either migr—ted tow—rd q @migr—tions —re performed through thetreeAD or ™opied upw—rd from q in response to — stri™tlyEseri—liz—˜le singleEo˜je™t re—d request @the response tr—vels upw—rd —long the treeD see ƒe™E
  45. 45. 26 ¡ Quiver on the Edge: Consistent Scalable Edge Servicespigure PFSF q loses ™ont—™t with p—rent p —nd its su˜tree dis™onne™tsF prepl—™es dis™onne™ted proxies in distQ —nd re™onstitutes the o˜je™t so b —ndd ™—n m—ke progressFtion PFSAF xote th—t in ™—se of multiple simult—neous dis™onne™tionsD onlyone ™onne™ted pro™ess|th—t whi™h h—s the o˜je™t in its dis™onne™ted ™hild9ssu˜tree|will re™onstitute the o˜je™t from its lo™—l ™opyD ˜e™oming the newhe—d of distQ‘id “F „he pseudo™ode th—t p exe™utes when its ™hild q dis™onne™ts is thechildDisconnected@q A routine in pigure PFTF ƒpe™i™—llyD p repl—™es —ll inEst—n™es of q in p:localQ‘id “ with itself —nd — ’noEop4 oper—tion to exe™uteon™e p o˜t—ins the o˜je™t @line V{W —nd IP{IQAF es su™hD —ny retrieve requestth—t w—s initi—ted —t — ™onne™ted pro™ess —nd ˜lo™ked —t — dis™onne™tedproxy is now ˜lo™ked —t pD see pigure PFSE@˜AF por e—™h of these requests th—t—re now ˜lo™ked —t pD p ™re—tes —nd runEen—˜les — new thre—d @lines IH{II ofpigure PFTA to initi—te the migr—tion of p:objs‘id “ to the neigh˜or following@this inst—n™e ofA p in p:localQ‘id “D on™e p h—s the o˜je™tF sf the dis™onne™ted™hild w—s —t the he—d of p:localQ‘id “D then p re™onstitutes the o˜je™t simEply ˜y m—king its lo™—l ™opy @whi™h is the l—test —t —ny ™onne™ted pro™essA—v—il—˜le @lines S{TAF p —lso responds to —ny stri™tlyEseri—liz—˜le singleEo˜je™tre—d requests initi—ted ˜y — stillE™onne™ted pro™ess —nd forw—rded ˜y p to qD—nd for whi™h p h—s not o˜served — response @not shown in pigure PFTAF
  46. 46. 2.6 Object availability in dynamic conditions ¡ 27childDisconnected@qA GB snvoked when p9s ™hild q dis™onne™ts BGIF p:children 2 p:children n fqg GB ‚emove q —s — ™hild BGPF foreach id GB por e—™h o˜je™tFFFBGQF q0 2 p:localQ‘id “:head GB FFFs—ve the ™urrent he—d of localQ BGRF Qreplace@id ; qA GB FFFrun Qreplace for this o˜je™t BGSF if q0 a q GB sf q w—s the he—d ˜efore QreplaceFFF BGTF V @p:sem‘id “A GB FFFthen s —m the he—dY rele—se o˜je™t BGQreplace@id ; qA GB snvoked lo™—lly ˜y p BGUF foreach i a I; : : : ; p:localQ‘id “:size   I GB por e—™h element of localQD ex™ept t—il BGVF if p:localQ‘id “:elmt‘i“ a hq; £i GB sf it points to q @’£4 is wildE™—rdAFFF BGWF p:localQ‘id “:elmt‘i“ 2 hp; kV @p:sem‘id “Aki GB FFF™h—nge it to point to myself BGIHF t 2 new thread@kP @p:sem‘id “AY doMigrate@id AkA GB FFFw—it for o˜je™t —nd then migr—te it BGIIF t:enable@A GB FFFrunEen—˜le this thre—d BGIPF if p:localQ‘id “:tail a hq; £i GB sf the t—il is the dis™onne™ted ™hildFFF BGIQF p:localQ‘id “:tail 2 hp; kV @p:sem‘id “Aki GB FFFthen just repl—™e it ˜y myself BG pigure PFTF his™onne™tionEh—ndling —t p2.6.2 Leavessn order to volunt—rily le—ve the treeD — proxy p must ensure th—t —ny o˜je™tsin the su˜tree rooted —t p —re still —™™essi˜le to ™onne™ted pro™essesD on™ep le—vesF purthermoreD outst—nding retrieve —nd @stri™tly seri—liz—˜leA re—drequests forw—rded through p must not ˜lo™k —s — result of p le—ving thetreeF sf p is — le—f nodeD then it serves —ny retrieve —nd stri™tly seri—liz—˜lere—d requests ˜lo™ked on itD migr—tes —ny o˜je™ts held —t p to its p—rent@ƒe™tion PFQFPAD for™es its upd—tes to ˜e dur—˜le @ƒe™tion PFRFPAD —nd dep—rtsFsf p is —n intern—l node then it for™es its upd—tes to ˜e dur—˜leD serves —nystri™tly seri—liz—˜le singleEo˜je™t re—d requestsD —nd then ™hooses one of its™hildren q to promoteF „he promotion upd—tes q9s st—te —™™ording to thest—te —t pD —nd noties other neigh˜ors of p —˜out q9s promotionF fefore promoting qD p noties its neigh˜ors @in™luding qA to tempor—rilyhold future mess—ges destined for pD until they —re notied ˜y q th—t q9spromotion is ™omplete @—t whi™h point they ™—n forw—rd those mess—ges toq —nd repl—™e —ll inst—n™es of p in their d—t— stru™tures with qAF p then sendsto q — promote mess—ge ™ont—ining p:parentD p:childrenD p:localQ‘ “D p:objs‘ “
  47. 47. 28 ¡ Quiver on the Edge: Consistent Scalable Edge Services@orD r—therD only those o˜je™t versions th—t q does not yet h—veA —nd p:DepsF‡hen q re™eives theseD it upd—tes its p—rentD ™hildrenD o˜je™ts —nd o˜je™tdependen™ies —™™ording to p9s st—teFpigure PFUF ueue mergeF ƒh—ded —nd unsh—ded elements —re in p—rent9s —nd™hild9s localQD respe™tivelyF h—shed —rrows —re from — skipped element to theelement —dded nextF „he interesting p—rt of q9s promotion is how it merges q:localQ‘id “ withp:localQ‘id “ for e—™h id D so th—t —ny outst—nding retrieve requests for idth—t were ˜lo™ked —t p or qD or simply forw—rded to other pro™esses ˜yp or q or ˜othD will m—ke progress —s usu—l when q9s promotion is ™omEpleteD see pigure PFUF pigure PFV presents the pseudo™ode used ˜y — proEmoted ™hild q to merge q:localQ‘id “ with its p—rent p9s p:localQ‘id “ for e—™hidentier id D —s the p—rent volunt—rily le—ves the servi™eF sn order to mergep:localQ‘id “ —nd q:localQ‘id “D q ˜egins with q:localQ‘id “ if its head points top —nd p:localQ‘id “ otherwiseF q —dds elements from the ™hosen queueD s—yp:localQ‘id “D to — newly ™re—ted mergedQ until —n inst—n™e of q is re—™hed@line IW of pigure PFVAD s—y —t the ith indexD iFeFD p:localQ‘id “:elmt‘i“ a qF „hemerge —lgorithm then skips this ith element —nd ˜egins to —dd elements fromq:localQ‘id “ until —n inst—n™e of p is foundF „his element is skipped —nd the—lgorithm swit™hes ˜—™k to p:localQ‘id “ —dding elements st—rting from the@i C IAst indexF „his —lgorithm ™ontinues until ˜oth queues h—ve ˜een ™omEpletely @ex™ept for the skipped elementsA —dded to mergedQ F efter mergingthe two queuesD q repl—™es —ll o™™urren™es of p in mergedQ ˜y itselfD usingQreplace@id ; pA dened in pigure PFTF et this pointD —ny outst—nding retrieve requests th—t were initi—ted ˜yp @represented ˜y inst—n™es of p in p:localQ‘id “A now —ppe—r —s initi—ted
  48. 48. 2.7 Correctness ¡ 29…pon re™eiving @promote X gParent ; siblings ; parentQ ‘ “; parentObjs ‘ “; parentDeps A GB wess—ge from the le—ving q:parent BGIF foreach id GB por e—™h o˜je™tFFFBGPF if parentObjs ‘id “:ver q:objs‘id “:ver GB sf the p—rent9s version is newer th—n mineFFF BGQF q:objs‘id “ 2 parentObjs ‘id “ GB FFFthen repl—™e my inst—n™e with p—rent9s BGRF mergedQ ‘id “ 2 Y GB ƒt—rt with — fresh mergedQ BGSF if q:localQ‘id “:head a hq:parent; £i GB sf the he—d of my localQ points to p—rentFFF BGTF doMerge@q:localQ‘id “; parentQ ‘id “; GB FFFthen st—rt with my localQ BG q; q:parent; mergedQ ‘id “AUF else GB sf the he—d does not point to p—rentFFF BGVF doMerge@parentQ ‘id “; q:localQ‘id “; GB FFFthen st—rt with p—rent9s localQ BG q:parent; q; mergedQ ‘id “AWF q:localQ‘id “ 2 mergedQ ‘id “ GB ƒet localQ to the newly ™re—ted mergedQ BGIHF Qreplace@id ; q:parentA GB ‚epl—™e p—rent with myself in new localQ BGIIF q:parent 2 gParent GB „he old gr—ndEp—rent is now my p—rent BGIPF q:children 2 @q:children ‘ siblings A n fqg GB yld si˜lings —re now my ™hildren BGIQF q:Deps 2 q:Deps ‘ parentDeps GB edd p—rent9s o˜je™t dependen™ies BGdoMerge@localQ; localQ ; p; p ; mergedQ A 0 0 GB snvoked lo™—llyY merges the two queues BGIRF while localQFhasElements@A GB sf the rst queue is not emptyFFF BGISF hr; prog i 2 localQ:removeFromHead@A GB FFFthen remove its he—d BGITF if r Ta p 0 GB sf he—d does not point to other pro™essFFF BGIUF mergedQ :addToTail@hr; prog iA GB FFFthen —dd it to the t—il of mergedQ BGIVF else GB sf he—d points to the other pro™essFFF BGIWF doMerge@localQ0 ; localQ; p0 ; p; mergedQ A GB FFFthen skip it —nd re™urse with other queue BG pigure PFVF €seudo™ode —t q for its promotion˜y q sin™e —ll inst—n™es of p from p:localQ‘id “ —re ™opied to mergedQ —ndthen repl—™ed ˜y qF ‚etrieve requests forw—rded through p ˜ut not q now—ppe—r —s forw—rded through qD —s —ll elements in p:localQ‘id “ —re —dded tomergedQ D ex™ept inst—n™es of q F ‚etrieve requests forw—rded through q —ndnot p —ppe—r —s ˜efore sin™e q:localQ‘id “ elements —re —ll —dded to mergedQ Dex™ept inst—n™es of pF pin—llyD requests forw—rded through ˜oth p —nd q now—ppe—r —s forw—rded through only qD due to skipping elements in p:localQ‘id “th—t point to q —nd vi™eEvers—F2.7 Correctness„his se™tion proves the ™orre™tness of uiver9s proto™ols for —™hieving seriE—liz—˜ility —nd stri™t seri—liz—˜ilityF
  49. 49. 30 ¡ Quiver on the Edge: Consistent Scalable Edge Services rf rfDe
  50. 50. nition 1 @re—dsEfromD   D   £ A. An operation op i re—ds from op j , de- 3 3 rf rfnoted op j   op i , if op i inputs an object instance produced by op j . 3  £ 3 rfdenotes the transitive closure of   . 3Lemma 1. Let op i and op j denote distinct operations that output objectinstances oi and oj , respectively, where oi :id a oj :id and oi :ver a oj :ver.Then there are no operations op k and op l (distinct or not) performed by rf rfconnected processes such that op i   £ op k and op j   £ op l . 3 3Proof. emong the ™onne™ted pro™essesD the localQFtail pointers implementthe errow proto™ol @hemmer —nd rerlihy ‘IWWV“AD —ugmented to —™™ount fordis™onne™tions —s des™ri˜ed in ƒe™tion PFTFIF „his proto™ol ensures th—t pero˜je™t identierD migr—tions —mong ™onne™ted pro™esses o™™ur seri—llyF ‡edo not re™ount the proof of this f—™t hereY interested re—ders —re referred toDeFgFD hemmer —nd rerlihy ‘IWWV“Y rerlihy et —lF ‘PHHI“Y uuhn —nd ‡—ttenEhofer ‘PHHR“F „his f—™t implies th—t there is — unique o˜je™t inst—n™e ˜e—ring— p—rti™ul—r identier —nd version num˜er th—t is migr—ted ˜y ™onne™tedpro™essesF es — resultD the existen™e of two o˜je™t inst—n™es oi —nd oj with thes—me o˜je™t identier —nd version num˜er implies th—t —t le—st one of op i—nd op j D s—y op i D w—s performed ˜y — proxy th—t dis™onne™tsF woreoverD theproxy th—t performs op i must dis™onne™t prior to migr—ting oi @or h—vingit ™opied due to the migr—tionD ƒe™tion PFRD or the stri™tly seri—liz—˜le re—dDƒe™tion PFSFPD of —n o˜je™t inst—n™e th—t depends on oi A out of the su˜treeth—t dis™onne™tsF ytherwiseD the lowest ™onne™ted —n™estor in the treeD whore™onstitutes the o˜je™t following the dis™onne™tionD would re™onstitute oior — l—ter version @see ƒe™tion PFTFIAF ƒoD oi is never visi˜le in the ™onne™ted™omponent ™ont—ining the serverF „his —lso implies th—t for e—™h o˜je™t inEst—n™e o su™h th—t o A oi @o depends on oi AD o is not visi˜le in the ™onne™ted™omponentX if o is migr—ted @or ™opiedA up to the ™onne™ted ™omponentDthen oi @or — l—ter versionA must ˜e ™opied —long with it @see ƒe™tion PFQFRAF„hereforeD none of the other o˜je™t inst—n™es produ™ed ˜y op i —re visi˜lein the ™onne™ted ™omponentD —s e—™h of these inst—n™es depends on oi F es
  51. 51. 2.7 Correctness ¡ 31— ™onsequen™eD none of the inst—n™es produ™ed ˜y op i is ever re—d ˜y — rf™onne™ted pro™ess —nd so op i T  £ op k F 3 vemm— I ensures th—t per o˜je™t identierD there is — unique sequen™e ofo˜je™t inst—n™es @ordered ˜y version num˜erA th—t —re visi˜le to ™onne™tedpro™essesF sn —dditionD vemm— I —lso provides —n —venue ˜y whi™h we ™—ndene the Durable set for our proto™olD iFeFD to ™onsist of those upd—te operE—tions th—t produ™e o˜je™t inst—n™es visi˜le to the ™onne™ted pro™esses —ndthose re—d oper—tions th—t o˜serve those o˜je™t inst—n™esFDe
  52. 52. nition 2 @DurableA. The set Durable is de
  53. 53. ned inductively to includeoperations according to the following two rules (and no other operations): @1 A If op i was executed at a connected process, then op i P Durable. rf @2 A If op i P Durable and op j   £ op i , then op j P Durable. 3 felow we prove th—t the oper—tions in Durable —re seri—liz—˜le when theupd—tes —nd multiEo˜je™t re—ds —re implemented —s in ƒe™tion PFR —nd singleo˜je™t re—ds —re implemented —s in ƒe™tion PFSFIF purthermore oper—tions inDurable —re stri™tly seri—liz—˜le for the other in™—rn—tion of our systemD iFeFDwhen the upd—tes —nd multiEo˜je™t re—ds —re implemented —s in ƒe™tion PFR—nd single o˜je™t re—ds —re implemented —s in ƒe™tion PFSFPF xote th—t ineither ™—seD oper—tions in Durable —re in f—™t dur—˜leD sin™e ’losing4 —n upEd—te ™ould viol—te seri—liz—˜ility or stri™t seri—liz—˜ilityF pin—llyD note th—tvemm— I holds for either in™—rn—tion of our systemF2.7.1 Proof of serializabilityMulti-version Serializability theory yur system m—int—ins multiple verEsions of the s—me o˜je™t —t the s—me time @—lthough not —t the s—me pro™essADtherefore we —rgue the seri—liz—˜ility of our —lgorithms using multiEversionseri—liz—˜ility theory @fernstein et —lF ‘IWVU“AF wultiEversion seri—liz—˜ilitytheory —llows us to —rgue the seri—liz—˜ility of — set of oper—tions throughthe —™y™li™ity of — p—rti™ul—r gr—phD ™—lled the multi-version serializationgraphF
  54. 54. 32 ¡ Quiver on the Edge: Consistent Scalable Edge Services verDe
  55. 55. nition 3 @version pre™eden™eD  3A.   The version precedence relation, verdenoted  3, is de
  56. 56. ned for operations as follows: For distinct operations  op i , op j and op k , let op k read an object instance oj produced by op j and op iproduce an object instance oi such that oi :id a oj :id. If oi :ver oj :ver then ver verop i  3 op j , otherwise op k  3 op i .    De
  57. 57. nition 4 @wultiEversion seri—liz—tion gr—phA. A multi-version serializa-tion graph of a set S of operations, denoted MVSG @S A, is a directed graphwhose nodes are operations in S and there is an edge from operation op i to rf veroperation op j if op i   op j or op i  3 op j or both. 3   sn order to prove th—t the set S of oper—tions is seri—liz—˜leD it is ˜othne™ess—ry —nd su™ient to prove th—t MVSG @S A is —™y™li™ ‘fernstein et —lFDIWVUD „heorem SFR“F ‡e prove the —™y™li™ity of MVSG @DurableA in two stepsX pirst we proveth—t its su˜gr—ph ™onsisting only of upd—te —nd multiEo˜je™t re—d oper—tions@—nd the ™orresponding edgesA is —™y™li™F ‡e then prove th—t —dding singleEo˜je™t re—d oper—tions —nd the ™orresponding edges to this —™y™li™ su˜gr—phdoes not introdu™e —ny ™y™lesF vet DurableH denote the su˜set of Durable ™onsisting only of upd—te —ndmultiEo˜je™t oper—tionsF sn order to prove the —™y™li™ity of MVSG @DurableH ADwe des™ri˜e — te™hnique to —ssign timest—mps to oper—tions in DurableH D —ndthen prove th—t —ll edges in MVSG @DurableH A —re in timest—mp orderF ƒin™etimest—mp order is —™y™li™D this proves the —™y™li™ity of MVSG @DurableH AFxote th—t these timest—mps serve only to —rgue —˜out the order of oper—tions—nd do not —dd fun™tion—lity to our —lgorithmsFAssigning timestamps vet ts @op A denote the timest—mp —ssigned to —n opEer—tion op F vet input @op A —nd output @op A denote the set of o˜je™t inst—n™esinput to —nd produ™ed ˜y oper—tion op D respe™tivelyF ‡e —ssign timest—mpsto upd—te —nd multiEo˜je™t oper—tions su™h th—t for e—™h p—ir of oper—tions rfop i —nd op j D if op i   op j then ts @op i A ts @op j AF „imest—mps with these 3properties ™—n ˜e —ssigned —s followsX ƒtore — timest—mp ts recent@oA fore—™h o˜je™t inst—n™e oF por e—™h upd—te or multiEo˜je™t re—d oper—tion op Ddene maxTs @op A —sX