

Data Locality, Latency and Caching: JSR-107 and the new Java Standard
 Early (Major) Problems:
 Winter 1997, high stakes problems encountered by those
Enterprises that trusted the “Web Appli...
 What is at stake for Caching Solution Providers and
their Customers?
Caching technology providers had a huge incentive t...
A Brief History of the Data Locality &
Latency Problem

Data Locality, Latency and Caching: JSR-107 and the new Java Stand...
 The very first data locality problem – Ma Bell
and the FCC assignment of Area Codes. 1947.
 NYC
 LA
 CHIC

212
213
31...
 The Data Locality Problem
 Big Decision: How much am I willing to “pay” to have
my data as close as possible to process...
 Moving the data further away from the processing
…

P1

P2

Local Unix Kernel (IPC
Messages, Semaphores, Shared Memory)
...
 A little further …

P1

P2

RDBMS

Local Client-Server

Data Locality, Latency and Caching: JSR-107 and the new Java Sta...
 A little further … LAN Client Server
P1

P2

Client Host 1

Client Host 2

RDBMS

Data Server

Data Locality, Latency an...
 And further still … WAN Client Server
P1

P2

Internet

RDBMS

Public Data Source

Data Locality, Latency and Caching: J...
 Latency Problem = Given that data is available, how fast
can I access it for processing?

Forget networked client server...

Latency Math ….

Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

12

Latency Physics ….
 2,000 PhD Physicists now working on Wall St.
 U.S. Colleges and Universities cannot produce nearly...

Latency Physics ….
 Famous LL/HF Capital Markets Trading Algorithms =
 “Cherokee Nation”
 “Marco Polo”

 “Boston Shu...
 Why these names, e.g. Why “Cherokee Nation”?

Data Locality, Latency and Caching: JSR-107 and the new Java Standard

Latency Physics ….
 Bottom Line :


Great Latency iff Great
Location
 FAMOUS CASE STUDY = NYC

Carrier

Hotel
 Algor...
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

17
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

18
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

19
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

20
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

21
 Full story of the legend of the NYC Carrier Hotel … and
how it became the place where
 “wall street trading algorithms ...
 Beyond LL/HF Caching‟s Mathematics, Physics and
(dramatic) case studies we can sum up the general
definition of Caching ...
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

24
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

25
 CAP: The simplest view of simultaneous service
guarantee capabilities and tradeoffs.
Abstract Communication Partition

D...
 Ben starts a “Birthday Reminder” service
 @t=1 Call Ben … put(“name”,”DOB”);
 @t=2 Call Ben … get(„name”,”DOB”);
 //l...
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

28
 Protocols for Cache Replacement/Consistency
•
•
•
•
•
•
•

Belady‟s Algorithm (aka Clairvoyant)
LRU
MRU
Random
LFU
LIRRS...

Java Caching API “Join Points” =

Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

30
Enough of the theory of Caching… on to the
standard Java Caching API …

On to JSR-107!
Before JSR-107 all Java Caching ven...
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

32
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

33
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

34
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

35
 (blank)

Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

36
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

37
 A JSR-107 compliant Caching implementation is
“Transactional” in 2 important (different) senses:
 1.
Compound access/mu...
 Important to note re: JSR-107 specification of
“Transactional” Java Caching …
 If a Java Caching provider claims to be ...
 JSR-107 => all Java Caches now guarantee TX capabilities:

 @t=0

 @t=1

 TX_THR_1
utx.begin();

JCACHE_ENTRY
$5

mut...
Can we use JSR-107 API to prevent Dirty Reads?
Of course!!
Just code TX_THR_2 like this
javax.cache.CacheManager cm = java...
 JSR-107 => DIRTY READ prevention

 @t=0
 @t=1

 TX_THR_1
utx.begin();

JCACHE_ENTRY
$5

mutate(credit $1)

 @t=2
 @...
Can we use JSR-107 API to accommodate all TX_ISOLATION tolerances (and
intolerances) ?
 What about rare transactional use...
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

44
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

45
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

46
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

47
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

48
 Fluent Builder Pattern

Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

49
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

50
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

51
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

52
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

53
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

54
Data Locality, Latency and Caching: JSR-107 and the new Java Standard

2/15/2014

55

Questions (?)

 Acknowledgment: Much of this PPT was taken
from Greg Luck and Cameron Purdy‟s JavaOne
2012 presentation...
Upcoming SlideShare
Loading in …5
×

Data Locality, Latency and Caching: JSR-107 and the new Java JCACHE Standard

615 views

Published on

data locality latency and caching : JSR-107 and the new Java JCACHE standard

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
615
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Data Locality, Latency and Caching: JSR-107 and the new Java JCACHE Standard

  1. 1.  Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  2. 2.  Early (Major) Problems:  Winter 1997, high stakes problems encountered by those Enterprises that trusted the “Web Applications are ready!” promise.  CASE: The week before Christmas 1997 the web-facing OMS at www.amazon.com repeatedly crashed. Availability was severely impacted. Millions of dollars in potential revenue were lost. Credibility with customers damaged. Put simply, AMAZON.com cannot go down the week before Christmas!  An urgent, immediate solution was required to correct legacy application architectures so that they could reliably and performantly operate at “Internet scale”. Enter Caching Technology! Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  3. 3.  What is at stake for Caching Solution Providers and their Customers? Caching technology providers had a huge incentive to deliver a 100% sound and100% complete solution …. At Stake? The World Wide Web as a reliable and performant application platform. NEEDED: 1. Accommodate application architects & developers with the capability to rapidly implement Caching technology. 2. Offload the database (ULTRA URGENT!) 3. Scale UP (How do I make the cache‟s in-process heap scale?) 4. Scale OUT (How do I distribute my cache to multiple processes?) 5. Buffer against load variability Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  4. 4. A Brief History of the Data Locality & Latency Problem Data Locality, Latency and Caching: JSR-107 and the new Java Standard 4
  5. 5.  The very first data locality problem – Ma Bell and the FCC assignment of Area Codes. 1947.  NYC  LA  CHIC 212 213 312 …  MEMPHIS 901  RALEIGH 919 Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  6. 6.  The Data Locality Problem  Big Decision: How much am I willing to “pay” to have my data as close as possible to processing resources? Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 6
  7. 7.  Moving the data further away from the processing … P1 P2 Local Unix Kernel (IPC Messages, Semaphores, Shared Memory) Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 7
  8. 8.  A little further … P1 P2 RDBMS Local Client-Server Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 8
  9. 9.  A little further … LAN Client Server P1 P2 Client Host 1 Client Host 2 RDBMS Data Server Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 9
  10. 10.  And further still … WAN Client Server P1 P2 Internet RDBMS Public Data Source Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 10
  11. 11.  Latency Problem = Given that data is available, how fast can I access it for processing? Forget networked client server!! All Low-Latency apps use highly optimized data feeds to bring the data “local”. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 11
  12. 12.  Latency Math …. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 12
  13. 13.  Latency Physics ….  2,000 PhD Physicists now working on Wall St.  U.S. Colleges and Universities cannot produce nearly enough Physics PhDs to meet Wall St. demand  Many of these PhD Physicists are from former Soviet-Bloc countries  COLD-WAR priority work = building defenses that could stop Western advances in Stealth technology (largely unsuccessful)  POST COLD-WAR priority work = designing lowlatency, high-frequency Capital Markets trading algorithms (very successful!) Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 13
  14. 14.  Latency Physics ….  Famous LL/HF Capital Markets Trading Algorithms =  “Cherokee Nation”  “Marco Polo”  “Boston Shuffler”  “Frog Pond”  “Carnival”  “Twilight” Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 14
  15. 15.  Why these names, e.g. Why “Cherokee Nation”? Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  16. 16.  Latency Physics ….  Bottom Line :  Great Latency iff Great Location  FAMOUS CASE STUDY = NYC Carrier Hotel  Algorithms “check in” but … They don’t “check out” ! (details to follow)  http://www.carrierhotels.com/news/September200 1/60hudson.shtml Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 16
  17. 17. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 17
  18. 18. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 18
  19. 19. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 19
  20. 20. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 20
  21. 21. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 21
  22. 22.  Full story of the legend of the NYC Carrier Hotel … and how it became the place where  “wall street trading algorithms check in … but they don’t check out”  is very compellingly told here:  http://www.ted.com/talks/kevin_slavin_how_algorithms_sh ape_our_world.html Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 22
  23. 23.  Beyond LL/HF Caching‟s Mathematics, Physics and (dramatic) case studies we can sum up the general definition of Caching performance in one simple expression ratio:  How can we best make this happen w/ Java Caching technology? Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 23
  24. 24. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 24
  25. 25. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 25
  26. 26.  CAP: The simplest view of simultaneous service guarantee capabilities and tradeoffs. Abstract Communication Partition DataNode 1 DataNode 2  1. Allowing 1 or both available nodes to be UPDATED means you lose simultaneous Consistency  2. To guarantee simultaneous Consistency you must make 1 node unAvailable  3. If you want both C & A then all nodes must communicate and synchronize, but then you gain Partition risk (Communication could break). Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  27. 27.  Ben starts a “Birthday Reminder” service  @t=1 Call Ben … put(“name”,”DOB”);  @t=2 Call Ben … get(„name”,”DOB”);  //lacks availability … what if 2 calls arrive at same time?  Ben adds ex-wife to answer phones  Call svc … put(“name”,”DOB”);  Call svc … get(“name”,”DOB”);  //lacks consistency … what if call  @t=1 Ben answers = put(“name”,”DOB”)  @t=2 call back … but ..ex-Wife answers … get(“name”,”DOB”) fails?  Ben & ex-Wife add coherency and synchronization protocol  Call svc … put(“name”,”DOB”);  Ben & ex-Wife get in fight …stop talking. //coherency sol‟n useless!  Call svc … get(“name”,”DOB”);  //lacks partition tolerance Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 27
  28. 28. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 28
  29. 29.  Protocols for Cache Replacement/Consistency • • • • • • • Belady‟s Algorithm (aka Clairvoyant) LRU MRU Random LFU LIRRS CLOCK – FIFO/LIFO/FILO/LILO  Protocols for Cache Partition(Node) Health/Healing • • Cardiology Protocol Nodes obliged to produce “Heart Beat” • - • Gossip Protocol • • Nodes obliged to produce “rumors” about neighboring nodes Central “Gossip columnist” collect rumors, treats ‟illness‟ based on Gossip credibility policy. Central „Cardiologist‟ consumes HB pulses, treats „illness‟ Data Locality, Latency and Caching: JSR-107 and the new Java Standard
  30. 30.  Java Caching API “Join Points” = Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 30
  31. 31. Enough of the theory of Caching… on to the standard Java Caching API … On to JSR-107! Before JSR-107 all Java Caching vendors provided proprietary APIs Caching code suffered from vendor lock-in Just like JDBC solved this problem for the RDBMS community, JSR-107 solves this problem for the JCACHE community JSR-107 completely open … not hosted on private jcp.org EG forum. Hosted on http://groups.google.com/group/jsr107 public EG forum. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 31
  32. 32. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 32
  33. 33. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 33
  34. 34. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 34
  35. 35. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 35
  36. 36.  (blank) Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 36
  37. 37. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 37
  38. 38.  A JSR-107 compliant Caching implementation is “Transactional” in 2 important (different) senses:  1. Compound access/mutate operations against a Cache Entry are guaranteed to be EXCLUSIVE and ATOMIC.  (see javax.cache.Cache.EntryProcessor interface)  2. Compound operations against a Cache are guaranteed to preserve Transactional ACID properties.  (see JTA and javax.transaction.UserTransaction interface) Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 38
  39. 39.  Important to note re: JSR-107 specification of “Transactional” Java Caching …  If a Java Caching provider claims to be “JSR-107 Transactions compliant”, then all of the following transactional isolation levels are guaranteed to be available in the provider‟s JSR-107 implementation:  TX_READ_UNCOMMITTED  TX_READ_COMMITTED  TX_REPEATABLE_READ no „Dirty Read‟ guaranteed „Repeatable Read‟ assurance guaranteed  TX_SERIALIZABLE no „Phantom Read‟ guaranteed In other words, a 100% sound and 100% complete API capability for managing transaction isolation is guaranteed. CRUCIAL for Capital Markets Java Caching uses cases!! Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 39
  40. 40.  JSR-107 => all Java Caches now guarantee TX capabilities:  @t=0  @t=1  TX_THR_1 utx.begin(); JCACHE_ENTRY $5 mutate(credit $1)  @t=2  @t=3  @t=4  @t=5 TX_THR_2 utx.begin(); $6 $6 //more processing utx.rollback() val = access() //$6 $6 //more processing $5 $5 utx.commit() //$6 ??  // NOTE: @t=4 DIRTY_READ realized by TX_THR_2 access() @t=2 Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 40
  41. 41. Can we use JSR-107 API to prevent Dirty Reads? Of course!! Just code TX_THR_2 like this javax.cache.CacheManager cm = javax.cache.Caching.getCacheManager(id); javax.cache.Cache c = cm.createCacheBuilder(“bank_cache”) .setTransactionEnabled( javax.cache.transaction.IsolationLevel.READ_COMMITTED, javax.cache.transaction.Mode.LOCAL ).build(); javax.transaction.UserTransaction jsr107_utx = new JSR107ProviderJTAUserTransactionImpl(c); jsr107_utx.begin(); try { //process BigDecimal cash = c.get(ACCOUNT_ID); // @t=2 … guaranteed to BLOCK until TX_THR_1 commits/RB //process more jsr107_utx.commit(); } catch (Exception e) { jsr107_utx.rollback(); } // STILL NEEDED => OPTIMISITIC and PESSIMISTIC block policy (via API) for @t=2 DIRTY_READ Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 41
  42. 42.  JSR-107 => DIRTY READ prevention  @t=0  @t=1  TX_THR_1 utx.begin(); JCACHE_ENTRY $5 mutate(credit $1)  @t=2  @t=3  @t=4  @t=5 $6 $6 //more processing utx.rollback() TX_THR_2 utx.begin(); val = access() //BLOCK $6 $5 $5 val = access() //$5  // DIRTY READ guaranteed to be PREVENTED Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 42
  43. 43. Can we use JSR-107 API to accommodate all TX_ISOLATION tolerances (and intolerances) ?  What about rare transactional use cases that have extreme TX_ISOLATION demands? E.g. Intolerance to PHANTOM_READS? Can JSR-107 accommodate these rare (but important ) uses cases?  Answer = YES!!  What about multiple, dynamic views of the same Cache with different (possibly extreme) TX_ISOLATION demands?  Answer = YES  EG common Bond Portfolio CACHE shared by both RETAIL and INSTITUTIONAL Trading desks …   trade trade HY Credit Portfolio (3,000 bonds)      RETAIL desk 1000s of trades daily Each trade = few millis to transact Avg Trade = $5,000 USD Use JSR-107 TX_READ_COMMITTED       INSTITUTIONAL desk dozen or so trades daily Each trade = 4-6 secs to transact Avg Trade = $10,000,000 USD PHANTOM_READ intolerant Use JSR-107 TX_SERIALIZABLE Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 43
  44. 44. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 44
  45. 45. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 45
  46. 46. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 46
  47. 47. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 47
  48. 48. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 48
  49. 49.  Fluent Builder Pattern Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 49
  50. 50. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 50
  51. 51. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 51
  52. 52. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 52
  53. 53. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 53
  54. 54. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 54
  55. 55. Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 55
  56. 56.  Questions (?)  Acknowledgment: Much of this PPT was taken from Greg Luck and Cameron Purdy‟s JavaOne 2012 presentation on JSR-107 see=  http://www.parleys.com/#st=5&id=3046 Data Locality, Latency and Caching: JSR-107 and the new Java Standard 2/15/2014 56

×