Copyright	  Push	  Technology	  2012	  QCON	  London	  2013	  
InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and ...
Presented at QCon Londonwww.qconlondon.comPurpose of QCon- to empower software development by facilitating the spread ofkn...
Copyright	  Push	  Technology	  2012	  My	  ‘number’	  #6752	  
Copyright	  Push	  Technology	  2012	  Darach@PushTechnology.com	  About	  me?	  •  Distributed	  Systems	  /	  HPC	  guy....
Copyright	  Push	  Technology	  2012	  Highly	  Available	  Real-­‐Time	  Data	  Distribu`on	  at	  the	  network’s	  Edge...
Copyright	  Push	  Technology	  2012	  HA.	  Black	  swans	  &	  Pigfish!	  
Copyright	  Push	  Technology	  2012	  When	  this	  …	  Service	  esse	  hic	  draconum	  
Copyright	  Push	  Technology	  2012	  …	  inevitably	  happens	  HA!	  HA!	  W8!!	  
Copyright	  Push	  Technology	  2012	  Over	  the	  Edge,	  where	  Fallacies	  roam	  - The	  network	  is	  reliable	  -...
Copyright	  Push	  Technology	  2012	  &	  the	  last	  mile.	  Dragons	  there	  be.	  y	  =	  -­‐1.1667x	  +	  45703	  R...
Copyright	  Push	  Technology	  2012	  Distributed	  Systems	  “A	  distributed	  system	  is	  one	  in	  which	  the	  f...
Copyright	  Push	  Technology	  2012	  High	  Availability	  “The	  price	  of	  reliability	  is	  the	  pursuit	  of	  t...
Copyright	  Push	  Technology	  2012	  High	  Experiences	  ~	  Man	  vs	  Machine	  Image:	  hgp://www.phillymarke`nglabs...
Copyright	  Push	  Technology	  2012	  Machines	  are	  ‘Easy’	  A	  more	  unfortunate	  truth	  is:	  	  Humans*	  are	 ...
Copyright	  Push	  Technology	  2012	  Once	  Upon	  a	  *me	  …	  SPOFThere	  was	  a	  very	  useful	  ligle	  single	  ...
Copyright	  Push	  Technology	  2012	  It	  was	  so	  successful	  SPOFIt	  grew	  and	  it	  grew!	  So	  we	  put	  it	...
Copyright	  Push	  Technology	  2012	  It	  grew,	  and	  it	  grew	  …	  SPOFWe	  touched	  it	  with	  all	  of	  our	  ...
Copyright	  Push	  Technology	  2012	  See	  SPOF	  run!	  SPOFWe	  all	  touched	  it	  except	  Sergey	  ‘the	  emascula...
Copyright	  Push	  Technology	  2012	  See	  SPOF	  spew	  L	  Is	  there	  nothing	  we	  can	  do?	  Esse	  hic	  draco...
Copyright	  Push	  Technology	  2012	  Bad	  things	  happen	  …	  RapGenius	  vs	  Heroku.	  	  Anyone?	  Ferris?	  It’s	...
Copyright	  Push	  Technology	  2012	  So	  we	  spun	  up	  another	  instance	  And	  master	  /	  slave	  dabbled	  wit...
Copyright	  Push	  Technology	  2012	  Ac*ve/Passive	  –	  Maybe?	  MTTR	  rela`vely	  high.	  MTBF	  rela`vely	  low.	  	...
Copyright	  Push	  Technology	  2012	  Ac*ve/Passive	  –	  Hmm?	  MTTR	  slightly	  beger.	  Can	  replicate	  state	  ac`...
Copyright	  Push	  Technology	  2012	  Ac*ve/Ac*ve	  –	  Ugh…	  MTTR	  negligeable.	  Use	  2x	  the	  bandwidth.	  Get	  ...
Copyright	  Push	  Technology	  2012	  Ac*ve/Ac*ve	  –	  Ugh…	  MTTR	  negligeable.	  Minify	  bandwidth	  usage	  mostly....
Copyright	  Push	  Technology	  2012	  Ac*ve/Ac*ve	  –	  Ugh…	  MTTR	  negligeable.	  Minify	  bandwidth	  usage	  mostly....
Copyright	  Push	  Technology	  2012	  Spin	  up	  another	  box?	  Failback,	  Failover,	  Takeover?	  Load	  Balance?	  ...
Copyright	  Push	  Technology	  2012	  Can	  implement	  N-­‐Plex	  HA	  with	  a	  ‘simple’	  FSM	  Consider	  roles	  fo...
Copyright	  Push	  Technology	  2012	  Spin	  up	  a	  cluster,	  ring?	  MasterSlaveSlaveSlaveSlaveSlaveSlave SlaveA	  si...
Copyright	  Push	  Technology	  2012	  One	  more	  thing	  …	  Real-­‐Time	  streams	  Real-­‐`me	  data	  streams	  are	...
Copyright	  Push	  Technology	  2012	  Streaming	  Data	  Recovery	  hgp://nms.lcs.mit.edu/papers/ha-­‐icde05.pdf	  -­‐	  ...
Copyright	  Push	  Technology	  2012	  But,	  problem:	  The	  bird,	  basically.	  Immediately	  Inconsistent.	  But,	  E...
Copyright	  Push	  Technology	  2012	  Ugh,	  just	  one	  more	  thing	  …	  Data	  rarely	  passes	  through	  a	  box	 ...
Copyright	  Push	  Technology	  2012	  Streaming	  Opera*ons	  –	  4	  types	  hgp://nms.lcs.mit.edu/papers/ha-­‐icde05.pd...
Copyright	  Push	  Technology	  2012	  DOH?	  Data	  Recovery	  Type	  x	  Opera*on	  Type	  =	  The	  Hard	  Part	  
Copyright	  Push	  Technology	  2012	  Just	  one	  last	  one	  more	  thing	  …	  We	  are	  fickle,	  grumpy,	  changeab...
Copyright	  Push	  Technology	  2012	  Let’s	  see.	  Trading	  systems?	  There	  are	  at	  least	  26	  race	  condi`on...
Copyright	  Push	  Technology	  2012	  A	  most	  inconvenient	  truth	  So	  relax.	  Let	  it	  fail.	  Let	  live	  and...
Copyright	  Push	  Technology	  2012	  Stop.	  - The	  network	  is	  not	  reliable	  	  nor	  is	  it	  cost	  free.	  -...
Copyright	  Push	  Technology	  2012	  Look.	  - 	  High	  Availability	  is	  not	  enough	  for	  Machine	  to	  Human	 ...
Copyright	  Push	  Technology	  2012	  Listen.	  - 	  Every	  nuance	  comes	  with	  a	  set	  of	  tradeoffs.	  - 	  Choo...
Copyright	  Push	  Technology	  2012	  Ac`on:	  Data	  Distribu`on	  Messaging	  remixed	  around:	  	  Relevance 	  -­‐ 	...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Virtualize	  Client	  Queues	  BufferBloatProducers ConsumersABbabbNuance:...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Add	  data	  caching	  One	  hop	  closer	  to	  the	  edge	  …	  Is it ac...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Exploit	  data	  structure	  Snapshot	  recovery.	  Deltas	  or	  Changes	...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Behaviors	  Extensible.	  Nuance?	  Roll	  your	  own	  protocols.	  Trade...
Copyright	  Push	  Technology	  2012	  3rd	  Party	  Code	  in	  the	  Engine?	  •  A	  bug	  in	  3rd	  party	  code	  ca...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Structural	  Confla`on	  Ensures	  only	  current	  +	  consistent	  data	 ...
Copyright	  Push	  Technology	  2012	  Ac`on.	  Structural	  Confla`on	  [EEP]	  •  Replace/Overwrite	  ‘A1’	  with	  ‘A2’	...
Copyright	  Push	  Technology	  2012	  Client?	  Connec`on	  &	  Transport	  Cascading	  WebSockets Flashdiffusion.jsnativ...
Copyright	  Push	  Technology	  2012	  An	  extreme	  example.	  Box A Box B•  Ramp	  clients	  con`nuously	  •  100	  mes...
Copyright	  Push	  Technology	  2012	  Latency	  in	  perspec`ve	  •  2.4	  us.	  A	  tuned	  benchmark	  in	  C	  with	  ...
Copyright	  Push	  Technology	  2012	  Throughput.	  Server	  view.	  Non-­‐conflated	  data	  •  Cold	  start	  server	  •...
Copyright	  Push	  Technology	  2012	  Throughput.	  Client	  view.	  Non-­‐conflated	  data.	  •  What	  the	  ‘client’	  ...
Copyright	  Push	  Technology	  2012	  Throughput.	  Server	  view.	  Replace-­‐conflated	  •  Cold	  start	  server	  	  •...
Copyright	  Push	  Technology	  2012	  Throughput.	  Client	  view.	  Replace-­‐Conflated	  •  What	  the	  ‘client’	  sees...
Copyright	  Push	  Technology	  2012	  Can	  failure	  be	  fun?	  •  Where	  do	  HA	  algorithms	  fail	  humans?	  •  C...
Copyright	  Push	  Technology	  2012	  Who	  knows.	  Let’s	  figure	  it	  out!	  J	  HA!	  HA!	  W8!!	  
Copyright	  Push	  Technology	  2012	  My	  ‘number’	  #6752	  
Copyright	  Push	  Technology	  2012	  Darach@PushTechnology.com	  Ques`ons?	  •  Thank	  you	  for	  listening	  to,	  ha...
Upcoming SlideShare
Loading in …5
×

Highly Available Near Real-time Data Distribution Beyond the Network Edge

511 views

Published on

Video and slides synchronized, mp3 and slide download available at http://bit.ly/15ZiEO8.

Darach Ennis investigates data distribution biased for occasionally connected near-real-time data streaming in low fidelity environments with traditional messaging and discusses the nuances, tradeoffs and considerations that require a very different approach from traditional practices. Filmed at qconlondon.com.

Darach has a wealth of experience and expertise from his background working as Principal Consultant, Systems Engineer & Global Solutions Architect with companies such as Betfair, IONA, JP Morgan Chase and StreamBase. Regularly speaking at conferences and events on Big Data and CEP, Darach is a regular contributor to Big Data and CEP groups on LinkedIn.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
511
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Highly Available Near Real-time Data Distribution Beyond the Network Edge

  1. 1. Copyright  Push  Technology  2012  QCON  London  2013  
  2. 2. InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and BrazilianPortuguese)• Post content from our QCon conferences• News 15-20 / week• Articles 3-4 / week• Presentations (videos) 12-15 / week• Interviews 2-3 / week• Books 1 / monthWatch the video with slidesynchronization on InfoQ.com!http://www.infoq.com/presentations/real-time-data-distribution
  3. 3. Presented at QCon Londonwww.qconlondon.comPurpose of QCon- to empower software development by facilitating the spread ofknowledge and innovationStrategy- practitioner-driven conference designed for YOU: influencers ofchange and innovation in your teams- speakers and topics driving the evolution and innovation- connecting and catalyzing the influencers and innovatorsHighlights- attended by more than 12,000 delegates since 2007- held in 9 cities worldwide
  4. 4. Copyright  Push  Technology  2012  My  ‘number’  #6752  
  5. 5. Copyright  Push  Technology  2012  Darach@PushTechnology.com  About  me?  •  Distributed  Systems  /  HPC  guy.    •  Chief  Scien*st  :-­‐  at  Push  Technology  •  Alumnus  of  :-­‐      Motorola,  IONA,  BeTair,  JPMC,  StreamBase.  •  School:  Trinity  College  Dublin.    -­‐  BA  (Mod).  Comp.  Sci.+    -­‐  M.Sc.  Networks  &  Distributed  Systems      •  Responds  to:  Guinness,  Whisky  
  6. 6. Copyright  Push  Technology  2012  Highly  Available  Real-­‐Time  Data  Distribu`on  at  the  network’s  Edge  
  7. 7. Copyright  Push  Technology  2012  HA.  Black  swans  &  Pigfish!  
  8. 8. Copyright  Push  Technology  2012  When  this  …  Service  esse  hic  draconum  
  9. 9. Copyright  Push  Technology  2012  …  inevitably  happens  HA!  HA!  W8!!  
  10. 10. Copyright  Push  Technology  2012  Over  the  Edge,  where  Fallacies  roam  - The  network  is  reliable  - Latency  is  zero    - Bandwidth  is  infinite  - There  is  one  administrator  - Topology  does  not  change  - The  network  is  secure  - Transport  cost  is  zero  - The  network  is  homogeneous    hgp://en.wikipedia.org/wiki/Fallacies_of_Distributed_Compu`ng      
  11. 11. Copyright  Push  Technology  2012  &  the  last  mile.  Dragons  there  be.  y  =  -­‐1.1667x  +  45703  R²  =  0.02496  1.00E+00  1.00E+01  1.00E+02  1.00E+03  1.00E+04  sep./06   okt./06   okt./06   nov./06   dec./06   dec./06   jan./07   feb./07  Latency  Latency  (ms)  Expected  (ms)  Linear  (Latency  (ms))  -­‐10.00  0.00  10.00  20.00  30.00  sep./06   okt./06   okt./06   nov./06   dec./06   dec./06   jan./07   feb./07  Download  Bandwidth  Expected  (Mbps)  Download  (Mbps)  Buffer  Bloat   Cell  Coverage  Bagery!  
  12. 12. Copyright  Push  Technology  2012  Distributed  Systems  “A  distributed  system  is  one  in  which  the  failure  of  a  computer  you  didn’t  even  know  existed  can  render  your  computer  unusable”  -­‐  Leslie  Lamport  
  13. 13. Copyright  Push  Technology  2012  High  Availability  “The  price  of  reliability  is  the  pursuit  of  the  utmost  simplicity.”  -­‐  Sir  Tony  Hoare,  Turing  Award  speech,  1980      “This  is  the  unfortunate  truth:  Simplicity  is  Hard.”              -­‐    Mosely  &  Marks,  Out  of  the  Tar  Pit,  2006  
  14. 14. Copyright  Push  Technology  2012  High  Experiences  ~  Man  vs  Machine  Image:  hgp://www.phillymarke`nglabs.com/man-­‐vs-­‐machine/    
  15. 15. Copyright  Push  Technology  2012  Machines  are  ‘Easy’  A  more  unfortunate  truth  is:    Humans*  are  Harder.      Allegedly  Sergey*  can  be  Emasculated  by  Glass.  But,  I  bet  he  owns  a  god  phone.              *  Humans  are  grumpy!                                  *  Colors  chosen  at  random  
  16. 16. Copyright  Push  Technology  2012  Once  Upon  a  *me  …  SPOFThere  was  a  very  useful  ligle  single  point  of  failure.  
  17. 17. Copyright  Push  Technology  2012  It  was  so  successful  SPOFIt  grew  and  it  grew!  So  we  put  it  in  the  cloud.  
  18. 18. Copyright  Push  Technology  2012  It  grew,  and  it  grew  …  SPOFWe  touched  it  with  all  of  our  devices.  
  19. 19. Copyright  Push  Technology  2012  See  SPOF  run!  SPOFWe  all  touched  it  except  Sergey  ‘the  emasculated  one’  
  20. 20. Copyright  Push  Technology  2012  See  SPOF  spew  L  Is  there  nothing  we  can  do?  Esse  hic  draconum,  basically.  But,  yes  we  can  too  …  Wat?
  21. 21. Copyright  Push  Technology  2012  Bad  things  happen  …  RapGenius  vs  Heroku.    Anyone?  Ferris?  It’s  either  dyno  park  this  or  whale  fail  in  the  zoo  …  
  22. 22. Copyright  Push  Technology  2012  So  we  spun  up  another  instance  And  master  /  slave  dabbled  with  us  Master Slave
  23. 23. Copyright  Push  Technology  2012  Ac*ve/Passive  –  Maybe?  MTTR  rela`vely  high.  MTBF  rela`vely  low.    Ok  for    ‘nice-­‐to-­‐have’  or    ‘too-­‐late-­‐to-­‐cry-­‐now’    facili`es.    Active(Hot)Passive(Cold)
  24. 24. Copyright  Push  Technology  2012  Ac*ve/Passive  –  Hmm?  MTTR  slightly  beger.  Can  replicate  state  ac`vely  for  cheaper  recovery.  Active(Hot)Passive(Warm)
  25. 25. Copyright  Push  Technology  2012  Ac*ve/Ac*ve  –  Ugh…  MTTR  negligeable.  Use  2x  the  bandwidth.  Get  ‘half’  the  value.  Dups    Master SlaveActive(Hot)Active(Hot)
  26. 26. Copyright  Push  Technology  2012  Ac*ve/Ac*ve  –  Ugh…  MTTR  negligeable.  Minify  bandwidth  usage  mostly.    Dups    Active(Hot)Active(Warm)
  27. 27. Copyright  Push  Technology  2012  Ac*ve/Ac*ve  –  Ugh…  MTTR  negligeable.  Minify  bandwidth  usage  mostly.    Dups    Active(Hot)Active(Warm)
  28. 28. Copyright  Push  Technology  2012  Spin  up  another  box?  Failback,  Failover,  Takeover?  Load  Balance?  This  is  gesng  costly!  Active(Hot)Passive(Cold)Standby(Cold)Speak no evil Hear no evil See no evil
  29. 29. Copyright  Push  Technology  2012  Can  implement  N-­‐Plex  HA  with  a  ‘simple’  FSM  Consider  roles  for  Applica`on,  Controller,  callbacks,  …  InitialBootingBoot/DoBootSyncingSync/DoSyncStartSwitchingSwitching/DoSwitchStandbyStandby/DoSyncEndActiveActive/DoSwtichDoneSwtich/DoStandbyActivateStoppingStop/DoStopStartStandby/DoStopDone
  30. 30. Copyright  Push  Technology  2012  Spin  up  a  cluster,  ring?  MasterSlaveSlaveSlaveSlaveSlaveSlave SlaveA  simple  implementa`on  can  be  easier  to  build  than  HA  pairs.  Your  mileage  may  vary.  Can  be  masterless  too.      Where  possible,  use  exis`ng  solu`ons  such  as  Akka,  riak_core,  doozer,  Apache  Zookeeper.    You’ll  s`ll  need  to  roll  your  own  data  recovery  -­‐  mostly.  Although  CRDT’s  are  changing  that  in  the  database  community.  
  31. 31. Copyright  Push  Technology  2012  One  more  thing  …  Real-­‐Time  streams  Real-­‐`me  data  streams  are  long-­‐lived,  conversa`onal,  can  be  stateful.    Meh MehData!  
  32. 32. Copyright  Push  Technology  2012  Streaming  Data  Recovery  hgp://nms.lcs.mit.edu/papers/ha-­‐icde05.pdf  -­‐  HA  algos  for  DSPs    Type   Before   A]er   Comments  Precise   T1  T2  T3  T4  T5  T6  Completely  masks  failure.  Gap   T1  T2  T3  ?   T5  T6  May  result  in  data  loss  Repea*ng  Rollback   T1  T2  T3  T2  T3  T4  IO  preserva`on.  P/S  dups  are  equivalent.  Convergent  Rollback  T1  T2  T3  X2  X3  X4  IO  preserva`on.  P/S  dups  may  differ  ini`ally  but  EC.  Divergent  Rollback   T1  T2  T3  X2  X3  X  4  IO  preserva`on.  P/S  dups  will  differ  IIBECM
  33. 33. Copyright  Push  Technology  2012  But,  problem:  The  bird,  basically.  Immediately  Inconsistent.  But,  Eventually  Consistent  …  Maybe.        Humans  Hate  Repea*ng  Rollback  Recovery   IIBECM
  34. 34. Copyright  Push  Technology  2012  Ugh,  just  one  more  thing  …  Data  rarely  passes  through  a  box  without  being  fiddled  with.  State?  Finagle Finagle
  35. 35. Copyright  Push  Technology  2012  Streaming  Opera*ons  –  4  types  hgp://nms.lcs.mit.edu/papers/ha-­‐icde05.pdf  -­‐  HA  algos  for  DSPs    Type   Comments  Arbitrary   Completely  masks  failure.  Determinis*c   Given  Same  Input,  Same  State,  produce  Same  Output  Convergent-­‐capable   Given  Same  Input,  Earlier  Start,  Empty  State,  converge  to  Same  Output  as  Determinis`c  case  Repeatable   Given  Same  Input,  Earlier  State,  Empty  State,  requires  at  most  one  input  to  product  at  most  one  output  to  be  convergent-­‐capable  and  same  as  determinis`c  case  
  36. 36. Copyright  Push  Technology  2012  DOH?  Data  Recovery  Type  x  Opera*on  Type  =  The  Hard  Part  
  37. 37. Copyright  Push  Technology  2012  Just  one  last  one  more  thing  …  We  are  fickle,  grumpy,  changeable,  intolerant  …  Human  Expecta*ons  
  38. 38. Copyright  Push  Technology  2012  Let’s  see.  Trading  systems?  There  are  at  least  26  race  condi`ons  in    FIX  4.2  Order  Cancel/Replace/Reject  BrokerDealerServiceMarketDataFat CatBankerOrdersTradesMarketExecution"Flow"Prices PricesMarketDataMarketDataMarketDataMarketExecutionMarketExecutionMarketExecutionPrecise RecoveryGap RecoveryIt Depends…Its Complicated!And  please.  No  stale  data!  
  39. 39. Copyright  Push  Technology  2012  A  most  inconvenient  truth  So  relax.  Let  it  fail.  Let  live  and  let’s  learn.  Simple  is  Hard  
  40. 40. Copyright  Push  Technology  2012  Stop.  - The  network  is  not  reliable    nor  is  it  cost  free.  - Latency  is  not  zero  nor  is  it  a  democracy.  - Bandwidth  is  not  infinite  nor  predictable  especially  the  last  mile!  - There  is  not  only  one  administrator  trust,  rela`onships  are  key  - Topology  does  change  It  should,  however,  converge  eventually  - The  network  is  not  secure  nor  is  the  data  that  flows  through  it  - Transport  cost  is  not  zero  but  what  you  don’t  do  is  free  - The  network  is  not  homogeneous  nor  is  it  smart  
  41. 41. Copyright  Push  Technology  2012  Look.  -   High  Availability  is  not  enough  for  Machine  to  Human  -   High  Experience  requires  more  Data  has  a  *me  value  –  out  of  date  data  is  bad.  Data  has  variable  relevance,  based  on  loca*on,  device  …  Data  rates  are  faster  than  human  percep*on  You  cannot  send  100mb/sec  down  3G  networks  You  cannot  send  100mb/sec  down  3G  networks  You  cannot  …  waste  is  bad,  bloats  buffers,  and  ‘slows’  data.  Currency  &  (Immediate)  Consistency  are  important  in  M2H.  -   M2H  ‘High  Available  Experience’  might  work  for  M2M  …    
  42. 42. Copyright  Push  Technology  2012  Listen.  -   Every  nuance  comes  with  a  set  of  tradeoffs.  -   Choosing  the  right  ones  can  be  hard,  but  it  pays  off.  -   Context,  Environment  are  cri`cal  -   Break  all  the  rules,  one  benchmark  at  a  `me.    -   Benchmark  Driven  Development  
  43. 43. Copyright  Push  Technology  2012  Ac`on:  Data  Distribu`on  Messaging  remixed  around:    Relevance  -­‐  Queue  depth  for  conflatable  data  should  be  0  or  1.  No  more    Responsiveness  -­‐  Use  HTTP/REST  for  things.  Stream  the  ligle  things    Timeliness  -­‐  It’s  rela`ve.  M2M  !=  M2H.    Context    -­‐  Packed  binary,  deltas  mostly,  snapshot  on  subscribe.    Environment  -­‐  Don’t  send  1M  1K  events  to  a  mobile  phone  with  0.5mbps.  
  44. 44. Copyright  Push  Technology  2012  Ac`on.  Virtualize  Client  Queues  BufferBloatProducers ConsumersABbabbNuance:  Client  telemetry.  Tradeoff:  Durable  subscrip`ons  harder  
  45. 45. Copyright  Push  Technology  2012  Ac`on.  Add  data  caching  One  hop  closer  to  the  edge  …  Is it acache?Producers ConsumersABbabb xx
  46. 46. Copyright  Push  Technology  2012  Ac`on.  Exploit  data  structure  Snapshot  recovery.  Deltas  or  Changes  mostly.  Conserves  bandwidth  State!Producers ConsumersABbabb xxSnapshot Delta
  47. 47. Copyright  Push  Technology  2012  Ac`on.  Behaviors  Extensible.  Nuance?  Roll  your  own  protocols.  Tradeoff?  3rd  party  code  in  the  engine  :/  Thetopic isthecloud!Producers ConsumersABbabb xxX
  48. 48. Copyright  Push  Technology  2012  3rd  Party  Code  in  the  Engine?  •  A  bug  in  3rd  party  code  can  take  out  the  system.  •  Is  necessary  in  many  environments.  •  Can  force  low  density  deployments.  •  Solu`ons  are  appearing,  such  as  Waratek  for  the  JVM.  •  You  can  hive  out  3rd  party  code  into  a  lightweight  fully  isolated  container.  •  You  can  manage  the  container  lifecycle.  •  You  can  reallocate  memory,  CPU,  network.  •  It’s  language  level  virtualiza`on.  
  49. 49. Copyright  Push  Technology  2012  Ac`on.  Structural  Confla`on  Ensures  only  current  +  consistent  data  is  distributed.  Ac`vely  soaks  up  bloat.  Extensible!  Currentdata!Producers ConsumersABbabb xxX+
  50. 50. Copyright  Push  Technology  2012  Ac`on.  Structural  Confla`on  [EEP]  •  Replace/Overwrite  ‘A1’  with  ‘A2’  •  ‘Current  data  right  now’  •  Fast  •  Merge  A1,  A2  under  some  opera`on.  •  Opera`on  is  pluggable  •  Tunable  consistency  •  Performance  f(opera`on)  Replace   Merge  Cop Bop Aop ... ...C2 A2 C1 B1 A1 opC2 B1 A2C2 A2 C1 B1 A1 RC2 B1 A+C2 A2 C1 B1 A1 M+12345 5=A2 + A1=4 + 1112345 4158=C2 + C1=5 + 3Current  Current  +    Consistent  
  51. 51. Copyright  Push  Technology  2012  Client?  Connec`on  &  Transport  Cascading  WebSockets Flashdiffusion.jsnativeout of the box on the web and mobile web}Bespoke cascading -Have your cake and eat it
  52. 52. Copyright  Push  Technology  2012  An  extreme  example.  Box A Box B•  Ramp  clients  con`nuously  •  100  messages  per  second  per  client  •  Payload:  125  ..  2000  bytes  •  Message  style  vs  with  confla`on  •  Really  simple.  1  client  •  Ping  –  Pong.  Measure  RTT  `me.  •  Payload:  125  ..  2000  bytes  Throughput  (Worst  case)   Latency  (Best  case)  
  53. 53. Copyright  Push  Technology  2012  Latency  in  perspec`ve  •  2.4  us.  A  tuned  benchmark  in  C  with  low  latency  10Ge  NIC,  with  kernel  bypass,  with  FPGA  accelera`on  •  5.0  us.  A  basic  java  benchmark  –  as  good  as  it  gets  in  java  •  Diffusion  is  measurably  ‘moving  to  the  le•’  release  on  release  •  We’ve  been  ac`vely  tracking  and  con`nuously  improving  since  4.0  2.4 us 5.0 us 8.0 us 50.0 us~1000bytes(avg)~125bytes(avg)
  54. 54. Copyright  Push  Technology  2012  Throughput.  Server  view.  Non-­‐conflated  data  •  Cold  start  server  •  Ramp  750  clients  ‘simultaneously’  at  5  second  intervals    •  5  minute  benchmark  dura`on  •  Clients  onboarded  linearly  un`l  IO  (here)  or  compute  satura`on  occurs.    •  What  the  ‘server’  sees  
  55. 55. Copyright  Push  Technology  2012  Throughput.  Client  view.  Non-­‐conflated  data.  •  What  the  ‘client’  sees    •  At  and  beyond  satura`on  of  some  resource?  •  Things  break!    •  New  connec`ons  fail.  Good.  •  Long  established  connec`ons  ok.  •  Recent  connec`ons  `meout  and  client  connec`ons  are  dropped.  Good.  •  Diffusion  handles  smaller  message  sizes  more  gracefully    •  Back-­‐pressure  ‘waveform’  can  be  tuned  out.  Or,  you  could  use  structural  confla`on!  
  56. 56. Copyright  Push  Technology  2012  Throughput.  Server  view.  Replace-­‐conflated  •  Cold  start  server    •  Ramp  750  clients  ‘simultaneously’  at  5  second  intervals    •  5  minute  benchmark  dura`on  •  Clients  onboarded  linearly  un`l  IO  (here)  or  compute  satura`on  occurs.    •  Takes  longer  to  saturate  than  non-­‐conflated  case    •  Handles  more  concurrent  connec`ons    •  Again,  ‘server’  view  
  57. 57. Copyright  Push  Technology  2012  Throughput.  Client  view.  Replace-­‐Conflated  •  What  the  ‘client’  sees    •  Once  satura`on  occurs  Diffusion  adapts  ac`vely  by  degrading  messages  per  second  per  client    •  This  is  good.  Soak  up  the  peaks  through  fairer  distribu`on  of  data.    •  Handles  spikes  in  number  of  connec`ons,  volume  of  data  or  satura`on  of  other  resources  using  a  single  load  adap`ve  mechanism  
  58. 58. Copyright  Push  Technology  2012  Can  failure  be  fun?  •  Where  do  HA  algorithms  fail  humans?  •  Can  consistent  experiences  be  delivered  elapsing  short,  medium  and  longer  term  failures?  •  Does  an  M2H  eye  for  an  M2M  guy  even  work?    
  59. 59. Copyright  Push  Technology  2012  Who  knows.  Let’s  figure  it  out!  J  HA!  HA!  W8!!  
  60. 60. Copyright  Push  Technology  2012  My  ‘number’  #6752  
  61. 61. Copyright  Push  Technology  2012  Darach@PushTechnology.com  Ques`ons?  •  Thank  you  for  listening  to,  having  me    •  Le  twiger:  @darachennis    •  Work.  hgp://www.pushtechnology.com/  •  Fun.  hgp://github.com/darach/  •  Grab  me  any  `me  on  Weds/Thurs.    •  Like  Erlang?  Big  Data?  London  User  Group  meetup.  Thurs  March  7th,  QE2  18:30  –  20:00.  (Google  for  registra`on  details).      

×