2. whoami
Currently working as Senior Technologist @ Thomson Reuters
13 + years java development experience (portals, web services, content
management...)
3. Goals of this presentation
• Learn about caching/distributed caching basic
principles
• Get familiar with prominent use cases and the
list of solutions available in this space
• Be ‘Distributed cache’ related buzzwords
compliant
4. TwoHardThings
There are only two hard things in Computer
Science: cache invalidation and naming things.-
- Phil Karlton
No other efficient way to increase throughput/scalability than caching.
5. Why distributed Cache
2 minutes video from terracota that explains nicely about caching
(http://terracotta.org/video/)
6. Some fundamentals
• Cache
A temporary storage area where frequently accessed data can be stored for
rapid access. Defining frequently accessed data is a matter of judgment and
engineering
• Why Cache
The differences to fetch data from a CPU register , RAM, disk & network are
many orders of magnitude, so it makes perfect sense that keeping the most
frequently used data in the closest location that reduces latency. There is no
better way to improve the performance.
• Memory Hierarchy
Pictures of pyramids in the next slides should help to remember memory
architecture better
13. Distributed Cache
• One machine cannot manage huge amount of data
• 100s of servers need to be treated as single unit managing the
partitions, transaction, security & speed of concurrent access
• Distributed caching solutions simplifies the all the hard work required
by distributed programming
14. Shared Memory In Java
No support (direct) memory mapping (sharing memory across the different processes)
• Java is designed to be a (virtual) machine unto itself. It doesn't really support the idea of
separate processes. It has robust support for lightweight independent execution path
through threads sharing same memory space.
• Java's memory guarantees are a more fine-grained version of sharing memory, with type and
privacy control, and built-in robust concurrency features.
• There are ways access through new java nio APIs and through JNI.
Why memory mapped IO
• memory-mapped IO allows us to map a file on disk to memory byte buffer so that when we
read the memory buffer, the data is read from the file. There is no overhead of the system
call and the creation and copying of buffers. More importantly, from Java perspective, the
memory buffer resides in the native space and not in the JVM's heap.
Why its not so important for Java
• Java is a general purpose language, programmers are relieved from dealing with page faults,
in-appropriate access of disk sector & providing a layer over them memory
15. Use cases
• share data/state among many servers for better performance
• clustering of application
• partition your in-memory data
• send/receive messages among applications on demand
• distribute workload onto many servers
• take advantage of parallel processing
• provide fail-safe data management
• provide secure communication among servers
• better utilization of cpu and network bandwidth
16. & some economics
A blade with 64GB RAM for ~$1.5K
$30K we can have 1TB of RAM capacity (20 blades)
– Gartner estimates that by 2014 at least 40% of large organizations will
deploy an IMDG (In Memory Data Grid) product with the market
reaching to $1 billion
17. Caching topologies
- Partitioned
A partitioned cache is a clustered, fault-tolerant cache that
has linear scalability.
- Replication
A replicated cache is a clustered, fault tolerant cache where
data is fully replicated to every member in the cluster.
- Near Cache
A near cache is a hybrid cache; it typically fronts a distributed
cache or a remote cache with a local cache.
18. Cache Load techniques
• Cache Through
– Synchronous
• Write Behind
– Asynchronous
• Read Through (Lazy loading technique)
– if(get(Key)) is NULL load it.
Else return the result obtained from cache
19. Caching patterns
• Minimize the number of hops to locate and
access data
Separate data and metadata, provide hints, and
avoid cache-to-cache transfer
• Do not slow down - Cache data close to client
Location-hints
• Share data among many caches
Separate data paths and metadata paths,
location-hints and index
20. Cache Performance Characteristics
– Application throughput/latency
– JVM : Threads, Heap memory, GC
& also CPU, Memory, Disk at OS level
The main performance characteristic of a cache is a hit/miss ratio. The
hit/miss ratio is calculated as number of cache hits divided by number
of cache misses. The hit/miss ratio is calculated using hit and miss
counters accumulated over a period of time. A high hit/miss ratio
means that a cache performs well. A low hit/miss ratio means that the
cache is applied to data that should not be cached. Also, the low
hit/miss ratio may mean that a cache is too small to capture temporal
locality of data access.
21. Sample Usage (Hazelcast)
Map<String,User> users = new ConcurrentHashMap<String,User>();
users.put(“praveen", new AdminUser(“praveen", “yahoo123"));
users.put(“suresh", new ClientUser(“suresh", “wipro123"));
Single change will do the magic & we can get the users in
different JVMs and the host
Map<String,User> users = Hazelcast.getMap("users");
users.put(“praveen", new AdminUser(“praveen", “yahoo123"));
users.put(“suresh", new ClientUser(“suresh", “wipro123"));
22. Stand-alone JVM caching
JVM’s inbuilt mechanisms to handle the references are important concepts to
understand.
• Soft Reference
• Weak Reference
Ex: MapMarker library from Guava.
Map<Key,Graph> graphs = new MapMaker() .concurrencyLevel(4) .weakKeys() .maximumSize(10000)
.expireAfterWrite(10, TimeUnit.MINUTES) .makeComputingMap( new Function() {
public Graph apply(Key key) {
return createExpensiveGraph(key);
} });
