SlideShare a Scribd company logo
1 of 47
Download to read offline
Scaling Your Cache



          Alex Miller
          @puredanger
Mission

• Why does caching work?
• What’s hard about caching?
• How do we make choices as we
  design a caching architecture?
What is caching?
Lots of data
Memory Hierarchy
                                               Clock cycles to access

   Register   1


   L1 cache   3


   L2 cache   15


      RAM     200


       Disk   10000000


Remote disk   1000000000


          1E+00 1E+01 1E+02 1E+03 1E+04 1E+05 1E+06 1E+07 1E+08 1E+09
Facts of Life
  Register          Fast Small Expensive
 L1 Cache
 L2 Cache
Main Memory
 Local Disk
Remote Disk         Slow   Big   Cheap
Caching to the rescue!
Temporal Locality
Hits:                         0%
Cache:

Stream:
Temporal Locality
Hits:                         0%
Cache:

Stream:

Stream:

Cache:
Hits:                         65%
Non-uniform distribution
         Web page hits, ordered by rank
  3200                                      100%




  2400                                      75%




  1600                                      50%




   800                                      25%




    0                                        0%
              Page views, ordered by rank


               Pageviews per rank
               % of total hits per rank
Temporal locality
        +
  Non-uniform
  distribution
17000 pageviews
  assume avg load = 250 ms

cache 17 pages / 80% of views
   cached page load = 10 ms
    new avg load = 58 ms

    trade memory for
    latency reduction
The hidden benefit:
 reduces database load


Memory    Database
                                     ning
                                visio
                         r   pro
                  of ove
           line
A brief aside...


• What is Ehcache?
• What is Terracotta?
Ehcache Example

CacheManager manager = new CacheManager();
Ehcache cache = manager.getEhcache("employees");
cache.put(new Element(employee.getId(), employee));
Element element = cache.get(employee.getId());


   <cache   name="employees"
            maxElementsInMemory="1000"
            memoryStoreEvictionPolicy="LRU"
            eternal="false"
            timeToIdleSeconds="600"
            timeToLiveSeconds="3600"
            overflowToDisk="false"/>
Terracotta

App Node   App Node     App Node     App Node



           Terracotta   Terracotta
             Server       Server



App Node   App Node     App Node     App Node
But things are not
always so simple...
Pain of Large
   Data Sets
• How do I choose which
  elements stay in memory
  and which go to disk?
• How do I choose which
  elements to evict when I
  have too many?
• How do I balance cache size
  against other memory uses?
Eviction
When cache memory is full, what do I do?
• Delete - Evict elements
• Overflow to disk - Move to slower,
  bigger storage

• Delete local - But keep remote data
Eviction in Ehcache

Evict with “Least Recently Used” policy:
    <cache   name="employees"
             maxElementsInMemory="1000"
             memoryStoreEvictionPolicy="LRU"
             eternal="false"
             timeToIdleSeconds="600"
             timeToLiveSeconds="3600"
             overflowToDisk="false"/>
Spill to Disk in Ehcache
Spill to disk:
      <diskStore path="java.io.tmpdir"/>

      <cache   name="employees"
               maxElementsInMemory="1000"
               memoryStoreEvictionPolicy="LRU"
               eternal="false"
               timeToIdleSeconds="600"
               timeToLiveSeconds="3600"

               overflowToDisk="true"
               maxElementsOnDisk="1000000"
               diskExpiryThreadIntervalSeconds="120"
               diskSpoolBufferSizeMB="30" />
Terracotta Clustering
Terracotta configuration:
     <terracottaConfig url="server1:9510,server2:9510"/>

     <cache   name="employees"
              maxElementsInMemory="1000"
              memoryStoreEvictionPolicy="LRU"
              eternal="false"
              timeToIdleSeconds="600"
              timeToLiveSeconds="3600"
              overflowToDisk="false">

         <terracotta/>
     </cache>
Pain of Stale Data
• How tolerant am I of seeing
  values changed on the
  underlying data source?
• How tolerant am I of seeing
  values changed by another
  node?
Expiration

TTI=4


        0 1 2 3 4 5 6 7 8 9

TTL=4
TTI and TTL in Ehcache

 <cache   name="employees"
          maxElementsInMemory="1000"
          memoryStoreEvictionPolicy="LRU"
          eternal="false"
          timeToIdleSeconds="600"
          timeToLiveSeconds="3600"
          overflowToDisk="false"/>
Replication in Ehcache
<cacheManagerPeerProviderFactory
    class="net.sf.ehcache.distribution.
           RMICacheManagerPeerProviderFactory"
    properties="hostName=fully_qualified_hostname_or_ip,
                peerDiscovery=automatic,
                multicastGroupAddress=230.0.0.1,
                multicastGroupPort=4446, timeToLive=32"/>

<cache name="employees" ...>
    <cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory”
         properties="replicateAsynchronously=true,
         replicatePuts=true,
         replicatePutsViaCopy=false,
         replicateUpdates=true,
         replicateUpdatesViaCopy=true,
         replicateRemovals=true
         asynchronousReplicationIntervalMillis=1000"/>
</cache>
Terracotta Clustering
Still use TTI and TTL to manage stale data
between cache and data source

Coherent by default but can relax with
coherentReads=”false”
Write-Through Caching
                                               Ehcache 2.0
• Keep database in sync with database
• Cache write --> database write
• Ehcache 2.0 adds new API:
  •   Cache.putWithWriter(...)

  •   CacheWriter

      •   write(Element)

      •   writeAll(Collection<Element>)

      •   delete(Object key)

      •   deleteAll(Collection<Object> keys)
Write-Behind Caching
                                      Ehcache 2.0
• Allow database writes to lag cache updates
  • Improves write latency
  • Possibly reduces overall writes
• Use with read-through cache
• API same as write-through
• Other features:
  • Batching, coalescing, rate limiting, retry
Pain of Loading
• How do I pre-load the cache on startup?
• How do I avoid re-loading the data on every
  node?
Persistent Disk Store
<diskStore path="java.io.tmpdir"/>

<cache   name="employees"
         maxElementsInMemory="1000"
         memoryStoreEvictionPolicy="LRU"
         eternal="false"
         timeToIdleSeconds="600"
         timeToLiveSeconds="3600"
         overflowToDisk="true"
         maxElementsOnDisk="1000000"
         diskExpiryThreadIntervalSeconds="120"
         diskSpoolBufferSizeMB="30"

         diskPersistent="true" />
Bootstrap Cache Loader

Bootstrap a new cache node from a peer:
       <bootstrapCacheLoaderFactory
             class="net.sf.ehcache.distribution.
                    RMIBootstrapCacheLoaderFactory"
             properties="bootstrapAsynchronously=true,
                         maximumChunkSizeBytes=5000000"
             propertySeparator=",” />




On startup, create background thread to pull
the existing cache data from another peer.
Terracotta Persistence
Nothing needed beyond setting up
Terracotta clustering.

Terracotta will automatically bootstrap:
- the cache key set on startup
- cache values on demand
Cluster Bulk Load
                           Ehcache 2.0
• Terracotta clustered caches only
• Performance
  • 6 nodes, 8 threads, 3M entries
  • Coherent: 211.9 sec (1416 TPS)
  • Bulk Load: 19.0 sec (15790 TPS) - 11.2x
Cluster Bulk Load
                              Ehcache 2.0

CacheManager manager = new CacheManager();
Ehcache cache = manager.getEhcache("hotItems");

// Enter bulk loading mode for this node
cache.setCoherent(false);

// Bulk load
for(Item item : getItemHotList()) {
  cache.put(new Element(item.getId(), item));
}

// End bulk loading mode
cache.setCoherent(true);
Pain of Concurrency
• Locking
• Transactions
Thread safety

• Cache-level operations are thread-safe
• No public API for multi-key or multi-
  cache composite operations

• Provide external locking
BlockingCache
• Cache decorator
• On cache miss, get()s block until
  someone writes the key to the cache
• Optional timeout
  CacheManager manager = new CacheManager();
  Ehcache cache = manager.getEhcache("items");
  manager.replaceCacheWithDecoratedCache(
      cache,
      new BlockingCache(cache));
SelfPopulatingCache
• Cache decorator, extends BlockingCache
• get() of unknown key will construct the
  entry using a supplied factory
• “Read through” caching

 CacheManager manager = new CacheManager();
 Ehcache cache = manager.getEhcache("items");
 manager.replaceCacheWithDecoratedCache(
   cache,
   new SelfPopulatingCache(cache, cacheEntryFactory));
JTA
                         Ehcache 2.0


• Cache acts as XAResource
• Works with any JTA Transaction Manager
• Autodetects JBossTM, Bitronix, Atomikos
• Transactionally move data between
  database, cache, queue, etc
Pain of Duplication
• How do I get failover capability while avoiding
  excessive duplication of data?
Partitioning + Terracotta
        Virtual Memory
•   Each node (mostly) holds data it has seen
•   Use load balancer to get app-level partitioning
•   Use fine-grained locking to get concurrency
•   Use memory flush/fault to handle memory
    overflow and availability
•   Use causal ordering to guarantee coherency
Pain of Ignorance
• Is my cache being used?   How much?
• Is caching improving latency?
• How much memory is my cache using?
Dev Console
• Ehcache console
  • See caches, configuration, statistics
  • Dynamic configuration changes
• Hibernate console
  • Hibernate-specific view of caches
  • Hibernate stats
Scaling Your Cache
Scalability Continuum
 causal
ordering   YES NO                            NO                  YES                         YES
                                                                                  2 or more       2 or more
                                          2 or more JVMS                             JVMSmore
                                                                                     2 or
                                           2 or more big JVMs                                        2 or more
                                                                                                     JVMs
                     2 or more                                  2 or more              2 or more
                                                                                        JVMs           2 or more
                                                                                                        JVMs
                                                                                          JVMs of
                                                                                            lots
 # JVMs    1 JVM
                       2 or more
                        JVMS
                          JVMs
                                                                  2 or more
                                                                   JVMS
                                                                     JVMs                    JVMs
                                                                                                          JVMs of
                                                                                                            lots
                                                                                                             JVMs




                                                                  Terracotta
runtime    Ehcache
                       Ehcache
                         RMI
                                                Ehcache
                                               disk store            OSS            Terracotta FX         Terracotta FX
                                                                                        Ehcache FX           Ehcache FX




                            Ehcache DX                                         Ehcache EX and FX
                            management                                            management
                            and control                                           and control




                                                 more scale




                                                                                                     21
Thanks!

• Twitter - @puredanger
• Blog - http://tech.puredanger.com
• Terracotta - http://terracotta.org
• Ehcache - http://ehcache.org

More Related Content

What's hot

CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...
CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...
CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...smn-automate
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeperknowbigdata
 
Cassandra Day NY 2014: Getting Started with the DataStax C# Driver
Cassandra Day NY 2014: Getting Started with the DataStax C# DriverCassandra Day NY 2014: Getting Started with the DataStax C# Driver
Cassandra Day NY 2014: Getting Started with the DataStax C# DriverDataStax Academy
 
Above the clouds: introducing Akka
Above the clouds: introducing AkkaAbove the clouds: introducing Akka
Above the clouds: introducing Akkanartamonov
 
Administering and Monitoring SolrCloud Clusters
Administering and Monitoring SolrCloud ClustersAdministering and Monitoring SolrCloud Clusters
Administering and Monitoring SolrCloud Clusterslucenerevolution
 
Apache zookeeper seminar_trinh_viet_dung_03_2016
Apache zookeeper seminar_trinh_viet_dung_03_2016Apache zookeeper seminar_trinh_viet_dung_03_2016
Apache zookeeper seminar_trinh_viet_dung_03_2016Viet-Dung TRINH
 
Virtualizing Java in Java (jug.ru)
Virtualizing Java in Java (jug.ru)Virtualizing Java in Java (jug.ru)
Virtualizing Java in Java (jug.ru)aragozin
 
Bulk Loading into Cassandra
Bulk Loading into CassandraBulk Loading into Cassandra
Bulk Loading into CassandraBrian Hess
 
Concurrency in Scala - the Akka way
Concurrency in Scala - the Akka wayConcurrency in Scala - the Akka way
Concurrency in Scala - the Akka wayYardena Meymann
 
Cassandra 2.0 better, faster, stronger
Cassandra 2.0   better, faster, strongerCassandra 2.0   better, faster, stronger
Cassandra 2.0 better, faster, strongerPatrick McFadin
 
Bulk Loading Data into Cassandra
Bulk Loading Data into CassandraBulk Loading Data into Cassandra
Bulk Loading Data into CassandraDataStax
 
Spark / Mesos Cluster Optimization
Spark / Mesos Cluster OptimizationSpark / Mesos Cluster Optimization
Spark / Mesos Cluster Optimizationebiznext
 
Cassandra Community Webinar: Apache Cassandra Internals
Cassandra Community Webinar: Apache Cassandra InternalsCassandra Community Webinar: Apache Cassandra Internals
Cassandra Community Webinar: Apache Cassandra InternalsDataStax
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXzznate
 
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API ExamplesApache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API ExamplesBinu George
 

What's hot (20)

Ehcache 3 @ BruJUG
Ehcache 3 @ BruJUGEhcache 3 @ BruJUG
Ehcache 3 @ BruJUG
 
CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...
CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...
CocoaHeads PDX 2014 01 23 : CoreData and iCloud Improvements iOS7 / OSX Maver...
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
 
Cassandra Day NY 2014: Getting Started with the DataStax C# Driver
Cassandra Day NY 2014: Getting Started with the DataStax C# DriverCassandra Day NY 2014: Getting Started with the DataStax C# Driver
Cassandra Day NY 2014: Getting Started with the DataStax C# Driver
 
JahiaOne - Performance Tuning
JahiaOne - Performance TuningJahiaOne - Performance Tuning
JahiaOne - Performance Tuning
 
Above the clouds: introducing Akka
Above the clouds: introducing AkkaAbove the clouds: introducing Akka
Above the clouds: introducing Akka
 
Memcached Study
Memcached StudyMemcached Study
Memcached Study
 
Administering and Monitoring SolrCloud Clusters
Administering and Monitoring SolrCloud ClustersAdministering and Monitoring SolrCloud Clusters
Administering and Monitoring SolrCloud Clusters
 
Apache zookeeper seminar_trinh_viet_dung_03_2016
Apache zookeeper seminar_trinh_viet_dung_03_2016Apache zookeeper seminar_trinh_viet_dung_03_2016
Apache zookeeper seminar_trinh_viet_dung_03_2016
 
Curator intro
Curator introCurator intro
Curator intro
 
Virtualizing Java in Java (jug.ru)
Virtualizing Java in Java (jug.ru)Virtualizing Java in Java (jug.ru)
Virtualizing Java in Java (jug.ru)
 
Bulk Loading into Cassandra
Bulk Loading into CassandraBulk Loading into Cassandra
Bulk Loading into Cassandra
 
Concurrency in Scala - the Akka way
Concurrency in Scala - the Akka wayConcurrency in Scala - the Akka way
Concurrency in Scala - the Akka way
 
Hazelcast
HazelcastHazelcast
Hazelcast
 
Cassandra 2.0 better, faster, stronger
Cassandra 2.0   better, faster, strongerCassandra 2.0   better, faster, stronger
Cassandra 2.0 better, faster, stronger
 
Bulk Loading Data into Cassandra
Bulk Loading Data into CassandraBulk Loading Data into Cassandra
Bulk Loading Data into Cassandra
 
Spark / Mesos Cluster Optimization
Spark / Mesos Cluster OptimizationSpark / Mesos Cluster Optimization
Spark / Mesos Cluster Optimization
 
Cassandra Community Webinar: Apache Cassandra Internals
Cassandra Community Webinar: Apache Cassandra InternalsCassandra Community Webinar: Apache Cassandra Internals
Cassandra Community Webinar: Apache Cassandra Internals
 
Advanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMXAdvanced Apache Cassandra Operations with JMX
Advanced Apache Cassandra Operations with JMX
 
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API ExamplesApache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples
Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples
 

Viewers also liked

Innovative Software
Innovative SoftwareInnovative Software
Innovative SoftwareAlex Miller
 
Releasing Relational Data to the Semantic Web
Releasing Relational Data to the Semantic WebReleasing Relational Data to the Semantic Web
Releasing Relational Data to the Semantic WebAlex Miller
 
Stream Execution with Clojure and Fork/join
Stream Execution with Clojure and Fork/joinStream Execution with Clojure and Fork/join
Stream Execution with Clojure and Fork/joinAlex Miller
 
Project Fortress
Project FortressProject Fortress
Project FortressAlex Miller
 
Java Concurrency Gotchas
Java Concurrency GotchasJava Concurrency Gotchas
Java Concurrency GotchasAlex Miller
 
Cracking clojure
Cracking clojureCracking clojure
Cracking clojureAlex Miller
 
Clojure: The Art of Abstraction
Clojure: The Art of AbstractionClojure: The Art of Abstraction
Clojure: The Art of AbstractionAlex Miller
 
Visualising Data on Interactive Maps
Visualising Data on Interactive MapsVisualising Data on Interactive Maps
Visualising Data on Interactive MapsAnna Pawlicka
 
Concurrent Stream Processing
Concurrent Stream ProcessingConcurrent Stream Processing
Concurrent Stream ProcessingAlex Miller
 
Strange Loop Conference 2009
Strange Loop Conference 2009Strange Loop Conference 2009
Strange Loop Conference 2009Alex Miller
 
Tree Editing with Zippers
Tree Editing with ZippersTree Editing with Zippers
Tree Editing with ZippersAlex Miller
 
Clojure/West Overview (12/1/11)
Clojure/West Overview (12/1/11)Clojure/West Overview (12/1/11)
Clojure/West Overview (12/1/11)Alex Miller
 
Scaling Your Cache And Caching At Scale
Scaling Your Cache And Caching At ScaleScaling Your Cache And Caching At Scale
Scaling Your Cache And Caching At ScaleAlex Miller
 
Collections In Java
Collections In JavaCollections In Java
Collections In JavaBinoj T E
 
Collection Framework in java
Collection Framework in javaCollection Framework in java
Collection Framework in javaCPD INDIA
 
Marshmallow Test
Marshmallow TestMarshmallow Test
Marshmallow TestAlex Miller
 
Java Collection framework
Java Collection frameworkJava Collection framework
Java Collection frameworkankitgarg_er
 

Viewers also liked (20)

Innovative Software
Innovative SoftwareInnovative Software
Innovative Software
 
Releasing Relational Data to the Semantic Web
Releasing Relational Data to the Semantic WebReleasing Relational Data to the Semantic Web
Releasing Relational Data to the Semantic Web
 
Blogging ZOMG
Blogging ZOMGBlogging ZOMG
Blogging ZOMG
 
Stream Execution with Clojure and Fork/join
Stream Execution with Clojure and Fork/joinStream Execution with Clojure and Fork/join
Stream Execution with Clojure and Fork/join
 
Project Fortress
Project FortressProject Fortress
Project Fortress
 
Java Concurrency Gotchas
Java Concurrency GotchasJava Concurrency Gotchas
Java Concurrency Gotchas
 
Cracking clojure
Cracking clojureCracking clojure
Cracking clojure
 
Clojure: The Art of Abstraction
Clojure: The Art of AbstractionClojure: The Art of Abstraction
Clojure: The Art of Abstraction
 
Visualising Data on Interactive Maps
Visualising Data on Interactive MapsVisualising Data on Interactive Maps
Visualising Data on Interactive Maps
 
Concurrent Stream Processing
Concurrent Stream ProcessingConcurrent Stream Processing
Concurrent Stream Processing
 
Strange Loop Conference 2009
Strange Loop Conference 2009Strange Loop Conference 2009
Strange Loop Conference 2009
 
Tree Editing with Zippers
Tree Editing with ZippersTree Editing with Zippers
Tree Editing with Zippers
 
Clojure/West Overview (12/1/11)
Clojure/West Overview (12/1/11)Clojure/West Overview (12/1/11)
Clojure/West Overview (12/1/11)
 
Java collection
Java collectionJava collection
Java collection
 
Scaling Your Cache And Caching At Scale
Scaling Your Cache And Caching At ScaleScaling Your Cache And Caching At Scale
Scaling Your Cache And Caching At Scale
 
07 java collection
07 java collection07 java collection
07 java collection
 
Collections In Java
Collections In JavaCollections In Java
Collections In Java
 
Collection Framework in java
Collection Framework in javaCollection Framework in java
Collection Framework in java
 
Marshmallow Test
Marshmallow TestMarshmallow Test
Marshmallow Test
 
Java Collection framework
Java Collection frameworkJava Collection framework
Java Collection framework
 

Similar to Scaling Your Cache

Building low latency java applications with ehcache
Building low latency java applications with ehcacheBuilding low latency java applications with ehcache
Building low latency java applications with ehcacheChris Westin
 
Scale ColdFusion with Terracotta Distributed Caching for Ehchache
Scale ColdFusion with Terracotta Distributed Caching for EhchacheScale ColdFusion with Terracotta Distributed Caching for Ehchache
Scale ColdFusion with Terracotta Distributed Caching for EhchacheColdFusionConference
 
Cache is King: Get the Most Bang for Your Buck From Ruby
Cache is King: Get the Most Bang for Your Buck From RubyCache is King: Get the Most Bang for Your Buck From Ruby
Cache is King: Get the Most Bang for Your Buck From RubyMolly Struve
 
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICES
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICESSpring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICES
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICESMichael Plöd
 
Memcached Presentation @757rb
Memcached Presentation @757rbMemcached Presentation @757rb
Memcached Presentation @757rbKen Collins
 
Jug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsJug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsDavide Carnevali
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentalsChris Adkin
 
CHI - YAPC NA 2012
CHI - YAPC NA 2012CHI - YAPC NA 2012
CHI - YAPC NA 2012jonswar
 
CHI-YAPC-2009
CHI-YAPC-2009CHI-YAPC-2009
CHI-YAPC-2009jonswar
 
Caching with Memcached and APC
Caching with Memcached and APCCaching with Memcached and APC
Caching with Memcached and APCBen Ramsey
 
Where Django Caching Bust at the Seams
Where Django Caching Bust at the SeamsWhere Django Caching Bust at the Seams
Where Django Caching Bust at the SeamsConcentric Sky
 
Caching your rails application
Caching your rails applicationCaching your rails application
Caching your rails applicationArrrrCamp
 
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...Amazon Web Services
 
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached georgepenkov
 
Bottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPBottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPkatzgrau
 
Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011CodeIgniter Conference
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton insertsChris Adkin
 
Performance and predictability (1)
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)RichardWarburton
 

Similar to Scaling Your Cache (20)

Building low latency java applications with ehcache
Building low latency java applications with ehcacheBuilding low latency java applications with ehcache
Building low latency java applications with ehcache
 
Scale ColdFusion with Terracotta Distributed Caching for Ehchache
Scale ColdFusion with Terracotta Distributed Caching for EhchacheScale ColdFusion with Terracotta Distributed Caching for Ehchache
Scale ColdFusion with Terracotta Distributed Caching for Ehchache
 
Cache is King: Get the Most Bang for Your Buck From Ruby
Cache is King: Get the Most Bang for Your Buck From RubyCache is King: Get the Most Bang for Your Buck From Ruby
Cache is King: Get the Most Bang for Your Buck From Ruby
 
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICES
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICESSpring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICES
Spring One 2 GX 2014 - CACHING WITH SPRING: ADVANCED TOPICS AND BEST PRACTICES
 
Memcached Presentation @757rb
Memcached Presentation @757rbMemcached Presentation @757rb
Memcached Presentation @757rb
 
Jug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsJug Lugano - Scale over the limits
Jug Lugano - Scale over the limits
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentals
 
CHI - YAPC NA 2012
CHI - YAPC NA 2012CHI - YAPC NA 2012
CHI - YAPC NA 2012
 
CHI-YAPC-2009
CHI-YAPC-2009CHI-YAPC-2009
CHI-YAPC-2009
 
Caching with Memcached and APC
Caching with Memcached and APCCaching with Memcached and APC
Caching with Memcached and APC
 
Where Django Caching Bust at the Seams
Where Django Caching Bust at the SeamsWhere Django Caching Bust at the Seams
Where Django Caching Bust at the Seams
 
Caching your rails application
Caching your rails applicationCaching your rails application
Caching your rails application
 
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) |...
 
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
 
Azure appfabric caching intro and tips
Azure appfabric caching intro and tipsAzure appfabric caching intro and tips
Azure appfabric caching intro and tips
 
Bottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMPBottom to Top Stack Optimization with LAMP
Bottom to Top Stack Optimization with LAMP
 
Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011Bottom to Top Stack Optimization - CICON2011
Bottom to Top Stack Optimization - CICON2011
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton inserts
 
Cache is King
Cache is KingCache is King
Cache is King
 
Performance and predictability (1)
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)
 

More from Alex Miller

Java Collections API
Java Collections APIJava Collections API
Java Collections APIAlex Miller
 
Java Concurrency Idioms
Java Concurrency IdiomsJava Concurrency Idioms
Java Concurrency IdiomsAlex Miller
 
Design Patterns Reconsidered
Design Patterns ReconsideredDesign Patterns Reconsidered
Design Patterns ReconsideredAlex Miller
 
Exploring Terracotta
Exploring TerracottaExploring Terracotta
Exploring TerracottaAlex Miller
 
Actor Concurrency
Actor ConcurrencyActor Concurrency
Actor ConcurrencyAlex Miller
 
Java Concurrency Gotchas
Java Concurrency GotchasJava Concurrency Gotchas
Java Concurrency GotchasAlex Miller
 

More from Alex Miller (7)

Java Collections API
Java Collections APIJava Collections API
Java Collections API
 
Java Concurrency Idioms
Java Concurrency IdiomsJava Concurrency Idioms
Java Concurrency Idioms
 
Design Patterns Reconsidered
Design Patterns ReconsideredDesign Patterns Reconsidered
Design Patterns Reconsidered
 
Java 7 Preview
Java 7 PreviewJava 7 Preview
Java 7 Preview
 
Exploring Terracotta
Exploring TerracottaExploring Terracotta
Exploring Terracotta
 
Actor Concurrency
Actor ConcurrencyActor Concurrency
Actor Concurrency
 
Java Concurrency Gotchas
Java Concurrency GotchasJava Concurrency Gotchas
Java Concurrency Gotchas
 

Recently uploaded

Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updateadam112203
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)codyslingerland1
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfTejal81
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosErol GIRAUDY
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 

Recently uploaded (20)

Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 update
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)
 
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdfQ4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
Q4 2023 Quarterly Investor Presentation - FINAL - v1.pdf
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenarios
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 

Scaling Your Cache

  • 1. Scaling Your Cache Alex Miller @puredanger
  • 2. Mission • Why does caching work? • What’s hard about caching? • How do we make choices as we design a caching architecture?
  • 5. Memory Hierarchy Clock cycles to access Register 1 L1 cache 3 L2 cache 15 RAM 200 Disk 10000000 Remote disk 1000000000 1E+00 1E+01 1E+02 1E+03 1E+04 1E+05 1E+06 1E+07 1E+08 1E+09
  • 6. Facts of Life Register Fast Small Expensive L1 Cache L2 Cache Main Memory Local Disk Remote Disk Slow Big Cheap
  • 7. Caching to the rescue!
  • 8. Temporal Locality Hits: 0% Cache: Stream:
  • 9. Temporal Locality Hits: 0% Cache: Stream: Stream: Cache: Hits: 65%
  • 10. Non-uniform distribution Web page hits, ordered by rank 3200 100% 2400 75% 1600 50% 800 25% 0 0% Page views, ordered by rank Pageviews per rank % of total hits per rank
  • 11. Temporal locality + Non-uniform distribution
  • 12. 17000 pageviews assume avg load = 250 ms cache 17 pages / 80% of views cached page load = 10 ms new avg load = 58 ms trade memory for latency reduction
  • 13. The hidden benefit: reduces database load Memory Database ning visio r pro of ove line
  • 14. A brief aside... • What is Ehcache? • What is Terracotta?
  • 15. Ehcache Example CacheManager manager = new CacheManager(); Ehcache cache = manager.getEhcache("employees"); cache.put(new Element(employee.getId(), employee)); Element element = cache.get(employee.getId()); <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="false"/>
  • 16. Terracotta App Node App Node App Node App Node Terracotta Terracotta Server Server App Node App Node App Node App Node
  • 17. But things are not always so simple...
  • 18. Pain of Large Data Sets • How do I choose which elements stay in memory and which go to disk? • How do I choose which elements to evict when I have too many? • How do I balance cache size against other memory uses?
  • 19. Eviction When cache memory is full, what do I do? • Delete - Evict elements • Overflow to disk - Move to slower, bigger storage • Delete local - But keep remote data
  • 20. Eviction in Ehcache Evict with “Least Recently Used” policy: <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="false"/>
  • 21. Spill to Disk in Ehcache Spill to disk: <diskStore path="java.io.tmpdir"/> <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="true" maxElementsOnDisk="1000000" diskExpiryThreadIntervalSeconds="120" diskSpoolBufferSizeMB="30" />
  • 22. Terracotta Clustering Terracotta configuration: <terracottaConfig url="server1:9510,server2:9510"/> <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="false"> <terracotta/> </cache>
  • 23. Pain of Stale Data • How tolerant am I of seeing values changed on the underlying data source? • How tolerant am I of seeing values changed by another node?
  • 24. Expiration TTI=4 0 1 2 3 4 5 6 7 8 9 TTL=4
  • 25. TTI and TTL in Ehcache <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="false"/>
  • 26. Replication in Ehcache <cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution. RMICacheManagerPeerProviderFactory" properties="hostName=fully_qualified_hostname_or_ip, peerDiscovery=automatic, multicastGroupAddress=230.0.0.1, multicastGroupPort=4446, timeToLive=32"/> <cache name="employees" ...> <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory” properties="replicateAsynchronously=true, replicatePuts=true, replicatePutsViaCopy=false, replicateUpdates=true, replicateUpdatesViaCopy=true, replicateRemovals=true asynchronousReplicationIntervalMillis=1000"/> </cache>
  • 27. Terracotta Clustering Still use TTI and TTL to manage stale data between cache and data source Coherent by default but can relax with coherentReads=”false”
  • 28. Write-Through Caching Ehcache 2.0 • Keep database in sync with database • Cache write --> database write • Ehcache 2.0 adds new API: • Cache.putWithWriter(...) • CacheWriter • write(Element) • writeAll(Collection<Element>) • delete(Object key) • deleteAll(Collection<Object> keys)
  • 29. Write-Behind Caching Ehcache 2.0 • Allow database writes to lag cache updates • Improves write latency • Possibly reduces overall writes • Use with read-through cache • API same as write-through • Other features: • Batching, coalescing, rate limiting, retry
  • 30. Pain of Loading • How do I pre-load the cache on startup? • How do I avoid re-loading the data on every node?
  • 31. Persistent Disk Store <diskStore path="java.io.tmpdir"/> <cache name="employees" maxElementsInMemory="1000" memoryStoreEvictionPolicy="LRU" eternal="false" timeToIdleSeconds="600" timeToLiveSeconds="3600" overflowToDisk="true" maxElementsOnDisk="1000000" diskExpiryThreadIntervalSeconds="120" diskSpoolBufferSizeMB="30" diskPersistent="true" />
  • 32. Bootstrap Cache Loader Bootstrap a new cache node from a peer: <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution. RMIBootstrapCacheLoaderFactory" properties="bootstrapAsynchronously=true, maximumChunkSizeBytes=5000000" propertySeparator=",” /> On startup, create background thread to pull the existing cache data from another peer.
  • 33. Terracotta Persistence Nothing needed beyond setting up Terracotta clustering. Terracotta will automatically bootstrap: - the cache key set on startup - cache values on demand
  • 34. Cluster Bulk Load Ehcache 2.0 • Terracotta clustered caches only • Performance • 6 nodes, 8 threads, 3M entries • Coherent: 211.9 sec (1416 TPS) • Bulk Load: 19.0 sec (15790 TPS) - 11.2x
  • 35. Cluster Bulk Load Ehcache 2.0 CacheManager manager = new CacheManager(); Ehcache cache = manager.getEhcache("hotItems"); // Enter bulk loading mode for this node cache.setCoherent(false); // Bulk load for(Item item : getItemHotList()) { cache.put(new Element(item.getId(), item)); } // End bulk loading mode cache.setCoherent(true);
  • 36. Pain of Concurrency • Locking • Transactions
  • 37. Thread safety • Cache-level operations are thread-safe • No public API for multi-key or multi- cache composite operations • Provide external locking
  • 38. BlockingCache • Cache decorator • On cache miss, get()s block until someone writes the key to the cache • Optional timeout CacheManager manager = new CacheManager(); Ehcache cache = manager.getEhcache("items"); manager.replaceCacheWithDecoratedCache( cache, new BlockingCache(cache));
  • 39. SelfPopulatingCache • Cache decorator, extends BlockingCache • get() of unknown key will construct the entry using a supplied factory • “Read through” caching CacheManager manager = new CacheManager(); Ehcache cache = manager.getEhcache("items"); manager.replaceCacheWithDecoratedCache( cache, new SelfPopulatingCache(cache, cacheEntryFactory));
  • 40. JTA Ehcache 2.0 • Cache acts as XAResource • Works with any JTA Transaction Manager • Autodetects JBossTM, Bitronix, Atomikos • Transactionally move data between database, cache, queue, etc
  • 41. Pain of Duplication • How do I get failover capability while avoiding excessive duplication of data?
  • 42. Partitioning + Terracotta Virtual Memory • Each node (mostly) holds data it has seen • Use load balancer to get app-level partitioning • Use fine-grained locking to get concurrency • Use memory flush/fault to handle memory overflow and availability • Use causal ordering to guarantee coherency
  • 43. Pain of Ignorance • Is my cache being used? How much? • Is caching improving latency? • How much memory is my cache using?
  • 44. Dev Console • Ehcache console • See caches, configuration, statistics • Dynamic configuration changes • Hibernate console • Hibernate-specific view of caches • Hibernate stats
  • 46. Scalability Continuum causal ordering YES NO NO YES YES 2 or more 2 or more 2 or more JVMS JVMSmore 2 or 2 or more big JVMs 2 or more JVMs 2 or more 2 or more 2 or more JVMs 2 or more JVMs JVMs of lots # JVMs 1 JVM 2 or more JVMS JVMs 2 or more JVMS JVMs JVMs JVMs of lots JVMs Terracotta runtime Ehcache Ehcache RMI Ehcache disk store OSS Terracotta FX Terracotta FX Ehcache FX Ehcache FX Ehcache DX Ehcache EX and FX management management and control and control more scale 21
  • 47. Thanks! • Twitter - @puredanger • Blog - http://tech.puredanger.com • Terracotta - http://terracotta.org • Ehcache - http://ehcache.org