Your SlideShare is downloading. ×
Coherence Implementation Patterns - Sig Nov 2011
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Coherence Implementation Patterns - Sig Nov 2011

7,685
views

Published on

Notes from the Nov Coherence Sig

Notes from the Nov Coherence Sig

Published in: Technology

0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,685
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
62
Comments
0
Likes
7
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Coherence  Implementa/on   Pa1erns   Ben  Stopford   The  Royal  Bank  of  Scotland  
  • 2. Some  Ideas     Nothing  More  
  • 3. Why  do  we  use  Coherence?   Fast?   Scalable?   Applica/on  layer?  
  • 4. Simplifying  the  Contract  
  • 5. •  We  don’t  want  ACID  all  of  the  /me  •  We  want  to  pick  the  bits  we  need  when  we   need  them  •  We  want  to  use  the  context  of  our  business   requirement  to  work  our  way  around  the  ones   we  don’t  need.  
  • 6. Version  your  Objects  
  • 7. Why  do  we  care?  Without  versioning  it’s  a  free-­‐for-­‐all.    •  What  changed?  •  Was  something  overwri1en?  •  How  can  you  prevent  concurrent  updates?  •  What  did  the  system  look  like  10  seconds  ago.  •  How  can  I  provide  a  consistent  view?  •  How  to  I  ensure  ordering  of  updates  in  an   asynchronous  system?  
  • 8. Versioning  your  Objects   Versioned   Cache   A.1   A.2   B.1   C.1   C.2   D.1  
  • 9. Versioning  your  Objects   Cache  A.1   Coherence  Trigger  A.2  New  Version  =  Old  Version  +  1  ??  
  • 10. Running  a  Coherence  Filter  
  • 11. Using  Key-­‐Based  Access  
  • 12. Latest  /  Versioned  Pa1ern   Coherence  Trigger  Write   Versioned   Latest  Cache   Cache   A   A.1   B   A.2   C   B.1   D   C.1   C.2   D.1  
  • 13. Latest  /  Versioned  Pa1ern  Latest  Cache  Key  =  [Business  Key]  Versioned  Cache  Key  =  [Business  Key][Version]   !" !&" #" !&(" $" #&" %" $&" $&(" %&"
  • 14. Suffers  from  data  duplica/on  
  • 15. Latest  Marker  Pa1ern   Write   Versioned   Coherence  Trigger   Cache   A.1   A.L   B.L  Well  Known  Marker  Version   C.1   C.L   D.L  
  • 16. However  our  trigger  can’t  use   cache.put()   Why?  
  • 17. Need  to  consider  the  threading  model  
  • 18. So  we’ll  need  to  use  the  backing  map   directly  public  void  copyObjectToVersionedCacheAddingVersion(MapTrigger.Entry  entry)  {        MyValue  value  =  (MyValue)entry.getValue();        MyKey  versionedKey  =  (MyKey)value.getKey();          BinaryEntry  binary  =  (BinaryEntry)entry;        Binary  binaryValue  =  binaryEntry.getBinaryValue();          Map  map  =  binary.getContext().getBackingMap("VersionedCacheName");        map.put(toBinary(versionedKey),  binaryValue);  }  
  • 19. A  third  approach  
  • 20. The  Collec/ons  Cache   Trigger  Appends  to  Collec/on  collec/onsCache.put(key,  val);   A   [O1,  O2,  O3..]   B   [O1,  O2,  O3..]  collec/onsCache.invoke(key,   C   [O1,  O2,  O3..]   new  LastValueGe1er());   …or  override  backing  store   D   [O1,  O2,  O3..]   Collec/onsCache  
  • 21. So  we  have  3  pa1erns  for  managing   versioning  whilst  retaining  key   based  access  
  • 22. Using  versioning  to  manage   concurrent  changes   Mul/  Version  Concurrency  Control     (MVCC)  
  • 23. Cache   Version  1   Coherence  Trigger   Version  2  New  Version  =  Old  Version  +  1  ??  
  • 24. Concurrent  Object  Update  (2  Clients  update  the  same  object  at  the  same  /me)   Client1   Client2   A.1   A.2   A.2  
  • 25. Concurrent  Object  Update   (Client2  fails  to  update  dirty  object)   Client1   Client2   A.1   A.2   Versioned   Cache  
  • 26. Concurrent  Object  Update   (Client  2  updates  clean  object)   Client1   Client2   A.1   A.2   A.3   Versioned   Cache  
  • 27. So  a  concurrent  update  results  in   an  error  and  must  be  retried.   &()*+#$ &()*+%$ !"#$ !"%$
  • 28. What’s  going  to  happen  if  we  are   using  putAll?  
  • 29. Reliable  PutAll  We  want  putAll  to  tell  us  which  objects  failed  the  write  process  
  • 30. Reliable  PutAll   Node   Node  Client   Extend   Node   Node   Node   Node   Invocable:   •  Split  keys  by  member   Invocable:   •  Send  appropriate  values   •  Write  entries  to   to  each  member   backing  map  (we   •  Collect  any  excep/ons   use  an  EP  for   returned     this)  
  • 31. This  gives  us  a  reliable  mechanism  for  knowing  what  worked  and  what   failed  
  • 32. Synthesising  Transac/onality  
  • 33. The  Fat  Object  Method   Cache   A   B   C   D  
  • 34. The  Single  Entry  Point  Method   (objects  are  stored  separately)       Collocate  with   key  associa/on  
  • 35. All  writes  synchronize  on  the   primary  object.   EP  
  • 36. All  reads  synchronize  on  the   primary  object.   EP  
  • 37. Wri/ng  Orphaned  Objects   Write  read   entry  point   object  last   Write  orphaned  objects  first   This  mechanism  is  subtly  flawed  
  • 38. Reading  several  objects  as  an   atomic  unit   aka  Joins  
  • 39. The  trivial  approach  to  joins  Get   Get   Get   Get   Get   Get   Get  Cost   Ledger Source   Transac MTMs   Legs   Cost  Centers   Books   Books   -­‐/ons   Centers   Network Time  
  • 40. Server  Side,  Sharded  Joins   Use  KeyAssocia/on  to   keep  related  en//es   together  Orders   Shipping  Log   Common  Key  
  • 41. Server  Side,  Sharded  Joins   Transactions Aggregator joins Mtms data across cluster Cashflows
  • 42. So  we  have  a  set  of  mechanisms  for   reading  and  wri/ng  groups  of   related  objects.  
  • 43. Cluster  Singleton  Service  
  • 44. A  service  that  automa/cally  restarts   amer  failure  
  • 45. A  service  that  automa/cally  restarts   amer  failure  
  • 46. What  is  the  cluster  singleton  good  for  •  Adding  indexes  •  Loading  data  •  Keeping  data  up  to  date  •  Upda/ng  cluster  /me  •  You  can  probably  think  of  a  bunch  of  others   yourselves.  
  • 47. Code  for  Cluster  Singleton  //run in a new thread on every Cache Serverwhile (true) { boolean gotLock = lockCache.lock("singletonLock", -1); if (gotLock) { //Start singletons wait(); }}
  • 48. Implemen/ng  Consistent  Views   and  Repeatable  Queries  
  • 49. Bi-­‐temporal    public  interface  MyBusinessObject{        //data     Business        public  Date  getBusinessDate();   Time        public  Date  validFrom();   System        public  Date  validTo();   Time  }    
  • 50. Where  does  the  System  Time   come  from?  
  • 51. You  can’t  use  the  System.currentTimeMillis()  in  a   distributed  environment!  
  • 52. You  need  a  cluster  synchronised   clock  
  • 53. Repeatable  Time:  A  guaranteed  Tick   Write  first   Write  Time  Singleton     Service   Read  Time   Write  second   Replicated  Caches   (pessimis/c)  
  • 54. As  we  add  objects  we  /mestamp  them   with  Write  Time   Write  Time  Singleton     Service  
  • 55. When  we  read  objects  we  use  Read   Time  Singleton     Service   Read  Time  
  • 56. Repeatable  Time:  A  guaranteed  Tick   8   7   8   7   8   7   Write  first   Write  Time  Singleton     Service   Read  Time   Write  second   6   7   6   7   6   7   Replicated  Caches   (pessimis/c)  
  • 57. Event  Based  Processing   Async     Cachestore  cache.put(key,  val);   A   B   C   D  
  • 58. Messaging  as  a  System  of  Record  Messaging System (use Topics for scalability)
  • 59. Messaging  as  a  System  of  Record   Trigger  cache.put(key,  val);   A   B   JMS C   TOPIC   D  
  • 60. Easy  Grid  Implementa/on  in  GUIs  
  • 61. CQCs  on  a  CQC   GUI  JVM  CQC   CQC   Cache   Define  this  in  config  
  • 62. How  do  you  release  quickly  to  a   Coherence  cluster?  
  • 63. Rolling  Restart?  
  • 64. Disk-­‐Persist  
  • 65. Final  Thoughts  
  • 66. Data  is  the  most  important  commodity  that  you  have     Keep  it  safe  
  • 67. Use  a  Par//on  Listener  
  • 68. Have  Proac/ve  Monitoring  of   Memory  
  • 69. Version  your  Objects  
  • 70. Thanks  Slides  &  related  ar/cles  available  at:    h1p://www.benstopford.com