Caching for J2eeEnterprise Applications Debajani Mohanty
Caching – The Need Web applications are typically accessed by many concurrent users. Usually, the applications data is stored in a relational database or file system, and it takes time and costs overhead to access these data sources. Database-access bottlenecks can slow down or even crash the application if it receives too many simultaneous requests. Object caching is one technique that overcomes this problem. Caching is one of most important factors if we think about application performance. Most applications need a caching implementation to improve performance.
Advantages Caching is meant for High scalability and performance It reduces number of trips to the database or other data sources, such as XML databases or ERP (enterprise resource planning) legacy systems It avoids the cost of repeatedly recreating objects It shares objects between threads in a process and between processes It efficiently uses process resources
Disadvantages Memory Size: The cache may consume significant heap space in the application server. JVM memory size can become unacceptably huge if a lot of unused data is in the cache and not released from memory at regular intervals. Synchronization complexity: Depending on the kind of data, complexity increases because consistency between the cached datas state and the data sources original data must be ensured. Otherwise, the cached data can fall out of sync with the actual data, which leads to data inaccuracies. Durability: Cache invalidation/deletion on server crash
Caching by HttpSession The HttpSession object (in the servlet package) also allows objects to be cached, but lacks the concepts of sharing, invalidation, per-object expiration, automatic loading, or spooling, which are the essential elements of a caching framework. Also HttpSession object becomes quite heavy, and are not meant to store huge amount of data.
Caching by EJB EJB provide a way to store an object in memory and perform the object lookup based on a key. But none of these methods provide any mechanism for either removing the object from memory when its no longer needed or automatically creating the object when its accessed after expiration
Caching Algorithms Cache should be invalidated with some algorithm if not needed as they consume memory and got limited capacity. Few Examples: least frequently used (LFU), least recently used (LRU), most recently used (MRU), first in first out (FIFO), last access time, and object size.
FIFO (First In First Out) Items are added to the cache as they are accessed, putting them in a queue or buffer and not changing their location in the buffer; when the cache is full, items are ejected in the order they were added. Cache access overhead is constant time regardless of the size of the cache. The advantage of this algorithm is that it’s simple and fast; it can be implemented using just an array and an index. The disadvantage is that it’s not very smart; it doesn’t make any effort to keep more commonly used items in cache. Summary for FIFO: fast, not adaptive, not scan resistant.
LRU (Least Recently Used) Items are added to the cache as they are accessed; when the cache is full, the least recently used item is ejected. Cache access overhead is against constant time. This algorithm is simple and fast, and it has a significant advantage over FIFO in being able to adapt somewhat to the data access pattern; frequently used items are less likely to be ejected from the cache. The main disadvantage is that it can still get filled up with items that are unlikely to be reaccessed soon; in particular, it can become useless in the face of scans over a larger number of items than fit in the cache. Nonetheless, this is by far the most frequently used caching algorithm . Summary for LRU: fast, adaptive, not scan resistant
LFU (Least Frequently Used) Frequency of use data is kept on all items. The most frequently used items are kept in the cache. The advantage is that long term usage patterns are captured well, incidentally making the algorithm scan resistant as well; the disadvantage, besides the larger access overhead, is that the algorithm doesn’t adapt quickly to changing usage patterns, and in particular doesn’t help with temporally clustered accesses. Summary for LFU: not fast, captures frequency of use, scan resistant
Simple time-based expiration Data in the cache is invalidated based on absolute time periods. Items are added to the cache, and remains in the cache for a specific amount of time. Summary for Simple time-based expiration: Fast, not adaptive, not scan resistant.
Extended time-based expiration Data in the cache is invalidated based on relative time periods. Items are added to the cache, and remains in the cache until they are invalidated at certain points in time, such as every five minutes, each day at 12:00AM etc. Summary for Extended time-based expiration: Fast, not adaptive, not scan resistant.
TTL (Time to live) expiration Data in the cache is invalidated by specifying the amount of time the item is allowed to be idle in the cache after last access time .
Caching Application Architecture Diagram
Main elements of an object- caching framework A typical caching framework contains components such as a CacheObject, CacheObjectKey, Cac he, CacheManager, and a CacheLoader.
ICacheManager ICacheManager is the main interface (contract) that a client program uses to handle all the operations related to caching (i.e., storing, accessing, and releasing the data in the cache). The client program can be a JSP (JavaServer Pages) page, a Struts action class or a POJO (plain old Java object). This interface was created to hide all the caching implementation details from the client so if we needed to switch to a different third-party caching API in the future, we wouldnt need to change any of the client code.
BaseCacheManager BaseCacheManager is the main class in the Web portal caching framework. Its the base implementation of ICacheManager. This class was created to centralize all the cache-related methods in one class. Its designed as a singleton to ensure one and only one instance ofICacheManager is created in the servlet containers JVM. In a clustered environment where multiple Web server/servlet container instances accept Web requests, a separate ICacheManager instance will be created in each JVM. If we switch to a different caching API later, this is the only class that must be modified to work with the new cache API. Also, if we switch to a JCache-compliant caching implementation, the cache manager should require minimal changes.
ICacheLoader The ICacheLoader interface implements the actual data-access logic in the Web client. All client programs that need to use the caching mechanism must implement this interface. It has one method calledloadCacheObject() and takes two input parameters, a string to specify the cache region name and an object to specify the cache key. This way, the cache manager knows which client program to use (to execute loadCacheObject()) to reload the object in the cache when the cached data expires after the specified time-to-live has elapsed. It is good practice for the caching service to load objects automatically as needed rather than using the application to directly manage objects that use the cache. When an application directly manages objects, it uses the CacheAccess.put() method to insert objects into the cache. To take advantage of automatic loading, we instead use a CacheLoader object and implement its load method to put objects into the cache. Note that the caching framework does not handle the creation of objects that need to be cached in a Web application, i.e., the data-access logic that retrieves the data from the data source is not coded in the caching classes. It relies on the client program to define the actual data-access logic. Technologies like Java Data Objects (JDO) are typically used to encapsulate the data-access logic in an enterprise Web application.
ICacheKey The ICacheKey interface was created to hide the specific logic used to create a cache key. Sometimes the cache key may not be a simple string. It may be as complex as the combination of multiple objects, and getting these values from the data source involves not one, but several, lookup methods. In this case, ICacheKey can define all the complex logic involved in creating the cache key. This way, the cache-key creation logic is defined in a separate class. I wrote a sample class called TestCacheKey that implements this interface and overrides the getCacheKey() method to illustrate how to use this interface.
CacheRegion A CacheRegion is defined as an organizational namespace for holding a collection of cache objects. Objects with similar characteristics (such as time-to-live and business use) should be cached in the same cache region so they can all be invalidated simultaneously if needed. To eliminate any synchronization issues that could cause poor performance, I used a separate instance ofCache for each cache region.
Cache LoaderIn conjunction with eviction policies, acache loader allows a user to maintain abounded cache for a large backenddatastore. Frequently used data is fetchedfrom the datastore into the cache, and theleast used data is evicted, in order toprovide fast access to frequently accesseddata. With a cache loader, the Java objectcache automatically determines if anobject needs to be loaded into the cachewhen the object is requested.
Locking Two kinds of locking are normally adopted for Java cache framework: pessimistic locking and optimistically locking. Pessimistic locking is an approach where an entity is locked for the entire time during a transaction. A lock either limits or prevents other users from working with the entity. Optimistic lock, which involved versioning data and maintaining copies for each transaction, validating copies upon transaction commit with the cached entity. This approach led to a very highly concurrent setup for a read-heavy system where readers are never blocked by concurrent writers, and also overcame the potential for deadlocks which may occur in pessimistically locked cache framework.
Different Caching Technologies There are several: Terracotta (open source, based on Mozilla Public License); Oracle Coherence (formerly Tangosol Coherence; commercial; based on JSR 107, which was never adopted officially); GigaSpaces (commercial; based on JavaSpaces API, part of Jini); GridGain, which you mentioned (open source: LGPL); Memcached with a Java client library (open source: BSD License; EHCache (open source: Apache Software License; OSCache (open source: modified Apache License; and
Grid ComputingGrid Computing is a way to distributeyour caching and computationsacross multiple computers (nodes).They are split them into 2categories: Compute Grids and DataGrids.
Compute Grid Compute Grids allow you to take a computation, optionally split it into multiple parts, and execute them on different grid nodes in parallel. The obvious benefit here is that your computation will perform faster as it now can use resources from all grid nodes in parallel. One of the most common design patterns for parallel execution is MapReduce. However, Compute Grids are useful even if you dont need to split your computation - they help you improve overall scalability and fault-tolerance of your system by offloading your computations onto most available nodes. Some Compute Grid vendors: - GridGain - Professional Open Source - JPPF - Open Source
Data Grid Data Grids allow you to distribute your data across the grid. Most of us are used to the term Distributed Cache rather than Data Grid (data grid does sound more savvy though). The main goal of Data Grid is to provide as much data as possible from memory on every grid node and to ensure data coherency. Some Data Grid/Cache vendors: - Oracle Coherence - Commercial - GemStone - Commercial - GigaSpaces - Commercial - JBossCache - Professional Open Source - EhCache - Open Source
Writing a Price to GigaSpaces In “classic” JavaSpaces every object in the space needsto implement “Entry” –GigaSpaces have optimised this out in their most recent release // Find the spaceJavaSpace space = (JavaSpace)SpaceFinder.find(“jini://*/*/ mySpace”);space.write( myPrice, null, Integer.MAX_VALUE ); // null transactions Because Sun haven’t evolved Jini and JavaSpaces GigaSpaces (and others) have had to do the innovation
Reading a Price from GigaSpaces // Find the spaceJavaSpace space = (JavaSpace)SpaceFinder.find(“jini://*/*/mySpace ”);// Define and template of what we’re looking for...Price template = new Price ( “GBP/NOK”, “SPOT”, null );Object entry = space.read( template, null, 10000 ); // null transactionsPrice myPrice = ((Price) entry); Optimisations and exception handling were left out for clarity e.g. –We would normally use a “snapshot” of the template Notice how the query is done by an interface/template rather than adata-centric query –This is more in line with SOA, i.e. Service Oriented as opposed to data-oriented So how does GigaSpaces work?
Writing a Price to Coherence We don’t need to implement anything –Since the data is distributed is must of course be Serializable // Create / Find a cache (using the map interface)NamedCache map = CacheFactory.getCache(“myCache");// Write the Price to the cache... Map.put( trade.getKey(), trade ); Creating an index... // Create an index on the “key” map.addIndex(new ReflectionExtractor(“getKey”, false, null); As you can see this couldn’t be easier
Reading a Price from Coherence // Create / Find a cache (using the map interface) NamedCache map = CacheFactory.getCache(“myCache");// Retrieve the latest spot price for GBP/NOK Price myPrice = (Price ) map.get(“GBP/NOK-SPOT” ); At this level Coherence works in a similar way to GigaSpaces
References http://java.dzone.com/articles/compute-grids-vs-data-grids - Compute Grids vs. Data Grids http://jasonwordpress.wordpress.com/2010/03/27/the- strategies-of-using-java-caching-frameworks-to-increase- the-performance-of-j2ee-applications/ - The strategies of using Java caching frameworks http://www.javaworld.com/javaworld/jw-05- 2004/jw-0531-cache.html - J2EE object-caching frameworks - JavaWorld http://www.scribd.com/doc/193625/Caching-Grid-In-A- Nutshell - Caching Grid in a Nutshell