인메모리 데이터그리드와 인피니스팬 소개

1. 인메모리 클러스터 아키텍처
JBoss 인피니스팬으로 살펴보는
인메모리 데이터그리드 아키텍처
전재홍
Dec 2013

2. Agenda




In-Memory Data Grid
Infinispan
Case Study
References

3. In-Memory Data Grid

4. Distributed Cache





Fast access to data
Performance Boost
Eviction, Expiration
Scalability
High Availability

5. Distributed Cache
Node
Node
App
App
App
Distributed
Cache
Cluster
Node
Persistence
(Database)

6. In-Memory Data Grid
 Evolution of distributed caches
 Clustered by nature
 Scalability (with cost)
– Horizontal scalability
– Design for elasticity
 High Availability
– No Single Point of Failure
– Redundancy + Load Balance
 Querying
 Task Execution (Map/Reduce)

7. In-Memory Data Grid
Node
App
Node
App
In-Memory
Data Grid
App
Node
Persistence
(Database)

8. For What





Sharing data (session, app state)
Toolkit for clustering
Performance (caching, in-memory processing)
Scalability
Database on cloud

9. Products









Oracle Coherence
GridGain
HazelCast
IBM eXtreme Scale
GigaSpaces
VmWare GemFire
Terracotta
ScaleOut StateServer
JBoss Infinispan

10. Cache vs. Data Grid
 JSR 107 - Temporary Caching for the Java Platform
–
–
–
–
–
Basic interaction(read, write, expiry)
Transactions with JTA compatibility
Listener
Persistence: write-through, write-behind
Annotations
 JSR 347 - Data Grids for the Java Platform
–
–
–
–
–
–
–
–
Asynchronous, non-blocking API
Distributed code execution and map/reduce API
Group API for co-location
Annotations (CDI)
Eventually Consistent API
Querying
Configuration
javax.datagrid.*

11. Infinispan

12. Infinispan






Distributed In-memory key/value Data Grid/ Cache
org.infinispan.Cache Interface
Distributed as Library and Server (from 5.3)
High availability
Elasticity
Manageable
– RHQ, JON
 Open source
– Apache v2 License
DefaultCacheManager manager = new DefaultCacheManager();
// Cache<Integer, Ticket> cache = manager.getCache();
Cache<Integer, Ticket> cache = manager.getCache(“myCache”);

13. Architecture: Library
Infinispan as Library - Standalone
Infinispan
App
JVM
JCP-107 Style Cache
just cache with advantages: expiry, j2ee transaction

14. Architecture: (Clustered) Library
Infinispan as Library - Clustered
Infinispan
App
Cluster
JVM
Infinispan
Easy deployment
More features
Richer APIs
Programmatic/
Declarative
configuration
– Extendable/
embeddable
– Faster (API call)
–
–
–
–
App
JVM
Infinispan
 Use as library
App
JVM
Application doesn’t know it’s on cluster

15. Architecture: Server
Infinispan as Server - Clustered
 Use as server
Infinispan
JVM
App
App
 Good Isolation
Infinispan
– App doesn’t
affect cluster
Cluster
JVM
Infinispan
App
– Protocols: Memc
ached, REST, Hot
Rod, WebSocket
JVM
 Data tier shared
by multi-apps
– Non-java client:
C++, .NET, Ruby,
Python, Java
Remote or embedded? – amount of data, etcs

16. Architecture: Durability
An Example of Durability
 Durability
Infinispan
JVM
Cluster
Infinispan
JVM
Infinispan
JVM
Infinispan
JVM
Cluster
persistence
– By replication
– By persistence
– By replication to
other cluster
(topology aware)

17. Cluster Mode: Replication(복제)
Replication Mode
cache.put(K,V)
Cache on
Server 2 K,V
Cache on
Server 1 K,V
Cache on
Server 3 K,V
Cache on
Server 4 K,V

18. Cluster Mode: Distribution(분산)
Distribution Mode(numOwners=2)
cache.put(K,V)
Cache on
Server 1 K,V
cache.get(K,V)
Cache on
Server 2 K,V
Cache on
Server 3
Cache on
Server 4

19. Cluster Mode: Invalidation(무효화)
Invalidation Mode
cache.put(K,V2)
Cache on
Server 1 K,V2
Cache on
Server 2 K,V
Cache on
Server 3
Cache on
Server 4
DB

20. Key Features: Persistence
 Used for durability
 Cache Store - Persistence Storage
– File System, Cloud, Remote, JDBC, JPA, LevelDB, Cassandra,
– HBase, MongoDB, BerkeleyDB, JDBM, REST




CacheLoader, CacheStore(CacheWriter from 6.0)
Read-through, Write-through, write-behind
Store chain
Shared store

21. Key Features: Transactions
 JTA Transaction Support
 Support MVCC (Multi-Versioned Concurrency Control)
 Isolation Level
– READ_COMMITTED (default)
– REPEATABLE_READ
 Locking Mode
– Optimistic Lock (default)
– Pessimistic Lock
 Locking
– Lock timeout
– Lock striping

22. Key Features: Query
 JBoss Hibernate Search + Apache Lucene
 Query on values
 Index Directory
– Lucene Directory: in-memory, file system, JDBC
– Infinispan Directory
 Distributed queries

23. Distributed Execution
 Executes codes on distributed nodes
 Through a standard JDK ExecutorService interface
 Use DistributedCallable extends
java.util.concurrent.Callable

24. Key Features: Map/Reduce
 Based on Distributed Execution Framework
 Mapper, Reducer, Collator, MapReduceTask
public interface Mapper<KIn, VIn, KOut, VOut> extends Serializable {
void map(KIn key, VIn value, Collector<KOut, VOut> collector);
}
public interface Reducer<KOut, VOut> extends Serializable {
VOut reduce(KOut reducedKey, Iterator<VOut> iter);
}
public interface Callator<KOut, VOut, R> {
R collate(Map<KOut, VOut>);
}

25. Common Features




Listeners
Hibernate L2 Cache
Near/Local Cache
Serialization

26. JDG




Red Hat JBoss Data Grid
Infinispan-based
JON
All the benefits of
subscription, including
Red Hat world class
support and services

27. Case Study

28. Case Study: Session Clustering
 Store session information into cache
in Spring MVC Interceptor

29. Case Study: Session Clustering
Store session information into cache
in Spring Security Filter
-
SecurityContextRepository를 구현한
CacheSecurityContextRepository 작성
loadContext, saveContext를 오버라이드하여
인피니스팬 사용
Spring cache abstraction 사용

30. Use Cases: Storm Processing State Store
Infinispan Data Grid

31. Use Cases: Data Grid Platform
In-Memory 대용량 데이터 처리를 위한 아키텍쳐 구조 제시 및 적용 사례,
제 6회 한국 소프트웨어 아키텍트 대회 최우수상, 2013

32. References
github.com/datagrids/spec/wiki
groups.google.com/group/jsr347
groups.google.com/group/jsr107
www.sap.com/inmemory
red.ht/data-grid
architects.dzone.com/articles/me
mory-data-grids-explained
 www.gridgain.com/blog/fyi/cache
-data-grid-database/
 infinispan.org
 infinispan-ko.blogspot.com







33. Thank you!
http://www.slideshare.net/jaehongc