From I/O To RAM

Riding The Memory Bus
Ophir Radnitz

Agenda
RAM vs. IO
Tools of the Trade
Introducing Terracotta
RAM Patterns with Terracotta

Anatomy of an Application
Get a DB connection, start transactioning
Start a request
Network
Get a bunch of data
Access disk,
Serialization
Release resources
Render stuff

I/O Issues
Scalability issues
Excessive network chatter
How do you scale the database?
Only as fast as your slowest query
Implies serialization/deserialization

Bandwidths (1 of 2)

RAM vs. Disk
Internet: ~80ms

RAM Goodness
RAM = High Bandwidth & Low Latency
No serialization costs
Less network

Use Cases
Low level cache
SQL, HTTP
Application cache
Hibernate 2nd level cache
Short lived entities
Queues

Generic Tools
Memcached
A high-performance, distributed memory caching system
Used EVRYWHERE: Facebook, Twitter, Digg, Wikipedia, Slashdot, LiveJournal, Sourceforge etc.
Squid
Routing and load balancing
HTTP cache

Memcached Limitations for Java
Remote cache
Serialization
2Gb limit

Java In-Memory Tools
Prevayler, Space4J
Persist objects in memory, supports ACID and file system journaling
Db4O in memory, Perst
Object oriented database
Jofti
A high-performance object indexing and searching solution. Supports Map.

In-Memory Considerations
Reliability requires redundancy
And/or durability
In a cluster state must be synced
Requires state distribution

Java Distribution Tools
Coherence
GigaSpace
Ehcache
Infinispan
Terracotta
Hazelcast

Network-Attached Memory infrastructure for the JVM
Open source
(TPL, based on Mozilla License)
Current version: 3.0.1

Terracotta Customers

Highly Available
Highly Scalable
Avoids excessive state replication
Hub and spoke architecture

A Nice Diagram
Application
Application
Application
Terracotta
Server
Terracotta
Server
Terracotta
Server
Application
Application

How Terracotta Works
Application
Terracotta
Client
Bootjarinstrumenting your application
Terracotta Libraries
JVM
Terracotta
Server
TC handles
JVM locks and
data access

How Terracotta Works, take 2
Application
Application
Application
Terracotta Server
Reference
Actual instance

Terracotta Features
Cross-JVM object identity
Automatic Persistence to disk
Virtual Memory
spill heap to TC and/or to disk, transparently
Cluster profiling & visualization
JMX-based monitoring and management

Terracotta Advantages
Easy to integrate, Maven support
Great scalability
No proprietary API (optional)
Ready made integration modules
Great diagnostics
In/out of process

Terracotta Development Console

Integrating Terracotta
Download & extract
Write a tc-config.xml
Create a bootjar
Run your app with the bootjar
Add elements & refactor

TIM - Terracotta Integration Modules
Packaged functionality
Easy to integrate
Handles locks details
Examples:
Concurrent collections, Ehcache
Lucene/Compass, Spring Security
Pipes, Wicket, Tomcat, Jetty
Hibernate, JBoss, Spring etc.

Terracotta Elements
HTTP Session Replication
Cache Evictor
Queues
Master-Worker
Asynchronous Processor
Cluster membership events

Terracotta Patterns
Data Cache
Intermediate persistence
Queues / work distribution
Write behind to database

Data Cache Considerations
Figure out data lifecycle
Read-only, metadata, configuration, session
Where cache is applicable
Single point of update
Set eviction policies
LRU, FIFO

Data Cache
Session cache is easier, entity cache needs real distribution
Use a generic cache
ConcurrentMap + Map Evictor (+ Jofti)
Or a Hibernate 2nd level cache
ehcache, TC-Cache

Where Should My Objects Live?
Memory
Database
Appropriateness
dies quickly
stays forever
Data Lifetime

Intermediate Data Lifetime
Some data doesn’t have to be stored in the database
Session information
Messages / Tokens
Transaction
Unfinished stuff
Data Lifetime

TIM Pipes: Queues
Execute time consuming tasks asynchronously
In a reliable way
Load balanced
Examples:
Send emails, process images
Supports pub/sub, routing
No need for JMS

CommonJ – Work Distribution
A joint API spec by Oracle and IBM
Provides interfaces for:
Worker / WorkItem / WorkManager
Scheduler
etc…
TIM-Messaging provides an implementation
Pipes
Master / Worker

Write Behind to System Of Record
Asynchronously persist to DB
Work queue
“1.5 phase commit”
Flags objects as dirty
Avoids syncing dirty objects
Flushes objects
Flags objects as clean

Terracotta Drawbacks
Concurrency awareness
Massive scalability is a commercial feature
Requires another machine(s)
Sharing objects across applications requires configuration
TIMs documentation

Terracotta Pros
Concurrency awareness
Fits into your application
Provides great visibility
Very versatile usage
Scales massively
Doesn’t use serialization
Great community

Summary
Ask yourself:
Does it belong in the DB?
Do I have to go to the DB for that?
Do I have to do it now?

Terracotta Pros
Any Questions?