• Save
Keeping Infinispan In Shape: Highly-Precise, Scalable Data Eviction
Upcoming SlideShare
Loading in...5
×
 

Keeping Infinispan In Shape: Highly-Precise, Scalable Data Eviction

on

  • 2,623 views

Java Collections Framework represents one of the key building blocks of any Java application. Although the standard JDK devoted a lot of attention to developing a coherent and easy to use collections ...

Java Collections Framework represents one of the key building blocks of any Java application. Although the standard JDK devoted a lot of attention to developing a coherent and easy to use collections framework one important aspect remains overlooked – collection element eviction. Collection memory footprint can not grow indefinitely because we would eventually run out of memory; we either have to remove elements from a collection or somehow periodically evict certain elements according to a chosen eviction algorithm. Since day one eviction has been a key feature of Infinispan, and in the latest 4.1 release thanks to event update batching, Infinispan has reduced the eviction overhead to such an extent that it hardly affects application performance. On top of that, Infinispan implements LIRS, a more precise eviction algorithm compared to the traditional LRU, making it the first open source project to implement this revolutionary algorithm in the data grid space. In this session, Galder and Vladimir will present to the details behind these changes, performance measurements and third-party use case testimonies.

Statistics

Views

Total Views
2,623
Views on SlideShare
2,623
Embed Views
0

Actions

Likes
6
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Keeping Infinispan In Shape: Highly-Precise, Scalable Data Eviction Keeping Infinispan In Shape: Highly-Precise, Scalable Data Eviction Presentation Transcript

  • galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Keeping Infinispan In Shape: Highly-Precise, Scalable Data Eviction Galder Zamarreño & Vladimir Blagojevic Senior Engineer, Red Hat 7th October 2010, JUDCon - Berlin galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Who is Galder? • R&D engineer (Red Hat Inc): • Infinispan developer • JBoss Cache developer • Contributor and committer: • JBoss AS, Hibernate, JGroups, JBoss Portal,...etc • Blog: zamarreno.com • Twitter: @galderz galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Agenda • Eviction in: • Java Collections Framework • JBoss Cache • Infinispan 4.0 • New in Infinispan 4.1: • LIRS • Batching Updates galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Java Collections Framework • Key building block of any Java app • Introduced in Java 1.2 • Extended with concurrent collections in Java 1.5 • Collection element eviction ?? galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Collections cannot grow forever galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • If using a collection as cache • Forces clients to either: • Remove elements proactively • Or run a periodic cleanup process • Which can be a PITA... galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Eviction in JBoss Cache days galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • JBoss Cache eviction issues • Under heavy load, eviction queues could fill up - bottleneck • Possibly a side effect of MVCC’s non-blocking reads • Separating data into regions could alleviate the issue • But it’s really a hack! galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Original Infinispan 4.0 eviction • Tried a different approach: • Avoid using queues to hold events • Taking advantage of tree to map change: • Attempt to maintain map ordered as per eviction rules • Could we use ConcurrentSkipListMap ? • No. O(1) desired for all map operations galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Doubly Linked ConcurrentHashMap • Based on H.Sundell and P.Tsigas paper: • Lock-Free Deques and Doubly Linked Lists (2008) • With each cache access or update update links • Eviction just a matter of walking the linked list one way galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Issues under heavy load • Algorithm always uses same node (the tail) to append stuff • With high concurrency, CAS stress lead to loads of retrying • So much retrying, we’re getting infinite loops • Before 4.0.0.Final, reverted to a more conservative algorithm galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • 4.0.0.Final eviction galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • LRU eviction algorithm issues • Weak access locality • One-time accessed keys not evicted timely • In loops, soon to be accessed keys might get evicted first • With distinct access frequencies, frequently accessed keys can unfortunately get evicted • LRU’s working set limited to cache size galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Enter the room... LIRS • Eviction algorithm that can cope with weak access locality • Based on S.Jiang and X.Zhang’s 2002 paper: • LIRS: An efficient low inter-reference recency set replacement policy to improve buffer cache performance • LIRS based around two concepts: IRR and Recency galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • IRR and Recency • Inter-Reference Recency (IRR): • Number of other unique keys accessed between two consecutive accesses to same key • Recency (R) • Number of other unique keys accessed from last reference until now galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • How does LIRS work? • If key has a high IRR, it’s next IRR is likely to be high again • Keys with highest IRR are considered for eviction • Once IRR is out of date, we start relying on Recency • LIRS = Low Inter-reference Recency Set galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • How is LIRS implemented? • Low IRR (LIR) area • Holds hot keys! • High IRR (HIR) area • Holds recently accessed keys • Keys here might get promoted to LIR area galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Cache Hit - LIR area galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Cache Hit - LIR area galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Hit - HIR area and in LIR Q ... galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Hit - HIR area and in LIR Q galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Hit - HIR area and not in LIR Q galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Cache Miss galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Cache Miss galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Cache Miss galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • LIRS implementation hurdles • LIRS requires a lot of key shifting around • It can lead to high contention • Unless you can implement it in scalable way, it’s useless • Low contended way to implement a high precision eviction algorithm? Is it possible? galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Batching Eviction Updates • Original idea: X.Ding, S.Jiang and X.Zhang’s 2009 paper: • BP-Wrapper: A System Framework Making Any Replacement Algorithms (Almost) Lock Contention Free • Keeping cache access per thread in a queue • If queue reaches a threshold: • Acquire locks and execute eviction as per algorithm • Batching updates significantly lowers lock contention galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Batching Updates in Infinispan • Decided against recording access per thread • 100s threads could be hitting cache; some short lived • Created BoundedConcurrentHashMap • Based on Doug Lea's ConcurrentHashMap • Records accesses in a lock-free queue in each segment • When threshold passed, acquire lock for segment and evict galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Precision and Performance • Segment level eviction does not affect overall precision • Community member run some performance tests: • After swapping their own LRU cache with BoundedCHM: • Berlin SPARQL Benchmark performance increased 55-60% for both cold and hot caches galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Summary • In JBoss Cache, eviction can become a bottleneck • Infinispan 4.0 uses conservative eviction • Infinispan 4.1 has more precise eviction algorithm (LIRS) • Batching updates, present in 4.1, significantly lowers lock contention • Result = Highly-concurrent, highly-precise implicit eviction galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010
  • Questions? infinispan.org blog.infinispan.org twitter.com/infinispan #infinispan #judcon galder@jboss.org | twitter.com/galderz | zamarreno.com Thursday, October 7, 2010