인메모리 클러스터 아키텍처
JBoss 인피니스팬으로 살펴보는
인메모리 데이터그리드 아키텍처
전재홍
Dec 2013
Agenda





In-Memory Data Grid
Infinispan
Case Study
References
In-Memory Data Grid
Distributed Cache






Fast access to data
Performance Boost
Eviction, Expiration
Scalability
High Availability
Distributed Cache

Node
Node

App

App

App

Distributed
Cache
Cluster
Node

Persistence
(Database)
In-Memory Data Grid
 Evolution of distributed caches
 Clustered by nature
 Scalability (with cost)
– Horizontal scalabi...
In-Memory Data Grid

Node
App
Node
App

In-Memory
Data Grid

App
Node

Persistence
(Database)
For What






Sharing data (session, app state)
Toolkit for clustering
Performance (caching, in-memory processing)
S...
Products










Oracle Coherence
GridGain
HazelCast
IBM eXtreme Scale
GigaSpaces
VmWare GemFire
Terracotta
Sca...
Cache vs. Data Grid
 JSR 107 - Temporary Caching for the Java Platform
–
–
–
–
–

Basic interaction(read, write, expiry)
...
Infinispan
Infinispan







Distributed In-memory key/value Data Grid/ Cache
org.infinispan.Cache Interface
Distributed as Lib...
Architecture: Library
Infinispan as Library - Standalone

Infinispan

App

JVM

JCP-107 Style Cache
just cache with advant...
Architecture: (Clustered) Library
Infinispan as Library - Clustered
Infinispan

App

Cluster

JVM
Infinispan

Easy deploym...
Architecture: Server
Infinispan as Server - Clustered

 Use as server

Infinispan

JVM
App

App

 Good Isolation
Infinis...
Architecture: Durability
An Example of Durability

 Durability
Infinispan
JVM

Cluster
Infinispan
JVM

Infinispan
JVM

In...
Cluster Mode: Replication(복제)
Replication Mode
cache.put(K,V)

Cache on
Server 2 K,V

Cache on
Server 1 K,V

Cache on
Serv...
Cluster Mode: Distribution(분산)
Distribution Mode(numOwners=2)
cache.put(K,V)

Cache on
Server 1 K,V

cache.get(K,V)

Cache...
Cluster Mode: Invalidation(무효화)
Invalidation Mode
cache.put(K,V2)

Cache on
Server 1 K,V2

Cache on
Server 2 K,V

Cache on...
Key Features: Persistence
 Used for durability
 Cache Store - Persistence Storage
– File System, Cloud, Remote, JDBC, JP...
Key Features: Transactions
 JTA Transaction Support
 Support MVCC (Multi-Versioned Concurrency Control)
 Isolation Leve...
Key Features: Query
 JBoss Hibernate Search + Apache Lucene
 Query on values
 Index Directory
– Lucene Directory: in-me...
Distributed Execution
 Executes codes on distributed nodes
 Through a standard JDK ExecutorService interface
 Use Distr...
Key Features: Map/Reduce
 Based on Distributed Execution Framework
 Mapper, Reducer, Collator, MapReduceTask
public inte...
Common Features





Listeners
Hibernate L2 Cache
Near/Local Cache
Serialization
JDG





Red Hat JBoss Data Grid
Infinispan-based
JON
All the benefits of
subscription, including
Red Hat world class
...
Case Study
Case Study: Session Clustering

 Store session information into cache
in Spring MVC Interceptor
Case Study: Session Clustering
Store session information into cache
in Spring Security Filter
-

SecurityContextRepository...
Use Cases: Storm Processing State Store

Infinispan Data Grid
Use Cases: Data Grid Platform

In-Memory 대용량 데이터 처리를 위한 아키텍쳐 구조 제시 및 적용 사례,
제 6회 한국 소프트웨어 아키텍트 대회 최우수상, 2013
References
github.com/datagrids/spec/wiki
groups.google.com/group/jsr347
groups.google.com/group/jsr107
www.sap.com/inmemo...
Thank you!

http://www.slideshare.net/jaehongc
Upcoming SlideShare
Loading in …5
×

인메모리 클러스터링 아키텍처

1,672 views

Published on

인메모리 데이터그리드와 인피니스팬 소개

Published in: Technology, Education

인메모리 클러스터링 아키텍처

  1. 1. 인메모리 클러스터 아키텍처 JBoss 인피니스팬으로 살펴보는 인메모리 데이터그리드 아키텍처 전재홍 Dec 2013
  2. 2. Agenda     In-Memory Data Grid Infinispan Case Study References
  3. 3. In-Memory Data Grid
  4. 4. Distributed Cache      Fast access to data Performance Boost Eviction, Expiration Scalability High Availability
  5. 5. Distributed Cache Node Node App App App Distributed Cache Cluster Node Persistence (Database)
  6. 6. In-Memory Data Grid  Evolution of distributed caches  Clustered by nature  Scalability (with cost) – Horizontal scalability – Design for elasticity  High Availability – No Single Point of Failure – Redundancy + Load Balance  Querying  Task Execution (Map/Reduce)
  7. 7. In-Memory Data Grid Node App Node App In-Memory Data Grid App Node Persistence (Database)
  8. 8. For What      Sharing data (session, app state) Toolkit for clustering Performance (caching, in-memory processing) Scalability Database on cloud
  9. 9. Products          Oracle Coherence GridGain HazelCast IBM eXtreme Scale GigaSpaces VmWare GemFire Terracotta ScaleOut StateServer JBoss Infinispan
  10. 10. Cache vs. Data Grid  JSR 107 - Temporary Caching for the Java Platform – – – – – Basic interaction(read, write, expiry) Transactions with JTA compatibility Listener Persistence: write-through, write-behind Annotations  JSR 347 - Data Grids for the Java Platform – – – – – – – – Asynchronous, non-blocking API Distributed code execution and map/reduce API Group API for co-location Annotations (CDI) Eventually Consistent API Querying Configuration javax.datagrid.*
  11. 11. Infinispan
  12. 12. Infinispan       Distributed In-memory key/value Data Grid/ Cache org.infinispan.Cache Interface Distributed as Library and Server (from 5.3) High availability Elasticity Manageable – RHQ, JON  Open source – Apache v2 License DefaultCacheManager manager = new DefaultCacheManager(); // Cache<Integer, Ticket> cache = manager.getCache(); Cache<Integer, Ticket> cache = manager.getCache(“myCache”);
  13. 13. Architecture: Library Infinispan as Library - Standalone Infinispan App JVM JCP-107 Style Cache just cache with advantages: expiry, j2ee transaction
  14. 14. Architecture: (Clustered) Library Infinispan as Library - Clustered Infinispan App Cluster JVM Infinispan Easy deployment More features Richer APIs Programmatic/ Declarative configuration – Extendable/ embeddable – Faster (API call) – – – – App JVM Infinispan  Use as library App JVM Application doesn’t know it’s on cluster
  15. 15. Architecture: Server Infinispan as Server - Clustered  Use as server Infinispan JVM App App  Good Isolation Infinispan – App doesn’t affect cluster Cluster JVM Infinispan App – Protocols: Memc ached, REST, Hot Rod, WebSocket JVM  Data tier shared by multi-apps – Non-java client: C++, .NET, Ruby, Python, Java Remote or embedded? – amount of data, etcs
  16. 16. Architecture: Durability An Example of Durability  Durability Infinispan JVM Cluster Infinispan JVM Infinispan JVM Infinispan JVM Cluster persistence – By replication – By persistence – By replication to other cluster (topology aware)
  17. 17. Cluster Mode: Replication(복제) Replication Mode cache.put(K,V) Cache on Server 2 K,V Cache on Server 1 K,V Cache on Server 3 K,V Cache on Server 4 K,V
  18. 18. Cluster Mode: Distribution(분산) Distribution Mode(numOwners=2) cache.put(K,V) Cache on Server 1 K,V cache.get(K,V) Cache on Server 2 K,V Cache on Server 3 Cache on Server 4
  19. 19. Cluster Mode: Invalidation(무효화) Invalidation Mode cache.put(K,V2) Cache on Server 1 K,V2 Cache on Server 2 K,V Cache on Server 3 Cache on Server 4 DB
  20. 20. Key Features: Persistence  Used for durability  Cache Store - Persistence Storage – File System, Cloud, Remote, JDBC, JPA, LevelDB, Cassandra, – HBase, MongoDB, BerkeleyDB, JDBM, REST     CacheLoader, CacheStore(CacheWriter from 6.0) Read-through, Write-through, write-behind Store chain Shared store
  21. 21. Key Features: Transactions  JTA Transaction Support  Support MVCC (Multi-Versioned Concurrency Control)  Isolation Level – READ_COMMITTED (default) – REPEATABLE_READ  Locking Mode – Optimistic Lock (default) – Pessimistic Lock  Locking – Lock timeout – Lock striping
  22. 22. Key Features: Query  JBoss Hibernate Search + Apache Lucene  Query on values  Index Directory – Lucene Directory: in-memory, file system, JDBC – Infinispan Directory  Distributed queries
  23. 23. Distributed Execution  Executes codes on distributed nodes  Through a standard JDK ExecutorService interface  Use DistributedCallable extends java.util.concurrent.Callable
  24. 24. Key Features: Map/Reduce  Based on Distributed Execution Framework  Mapper, Reducer, Collator, MapReduceTask public interface Mapper<KIn, VIn, KOut, VOut> extends Serializable { void map(KIn key, VIn value, Collector<KOut, VOut> collector); } public interface Reducer<KOut, VOut> extends Serializable { VOut reduce(KOut reducedKey, Iterator<VOut> iter); } public interface Callator<KOut, VOut, R> { R collate(Map<KOut, VOut>); }
  25. 25. Common Features     Listeners Hibernate L2 Cache Near/Local Cache Serialization
  26. 26. JDG     Red Hat JBoss Data Grid Infinispan-based JON All the benefits of subscription, including Red Hat world class support and services
  27. 27. Case Study
  28. 28. Case Study: Session Clustering  Store session information into cache in Spring MVC Interceptor
  29. 29. Case Study: Session Clustering Store session information into cache in Spring Security Filter - SecurityContextRepository를 구현한 CacheSecurityContextRepository 작성 loadContext, saveContext를 오버라이드하여 인피니스팬 사용 Spring cache abstraction 사용
  30. 30. Use Cases: Storm Processing State Store Infinispan Data Grid
  31. 31. Use Cases: Data Grid Platform In-Memory 대용량 데이터 처리를 위한 아키텍쳐 구조 제시 및 적용 사례, 제 6회 한국 소프트웨어 아키텍트 대회 최우수상, 2013
  32. 32. References github.com/datagrids/spec/wiki groups.google.com/group/jsr347 groups.google.com/group/jsr107 www.sap.com/inmemory red.ht/data-grid architects.dzone.com/articles/me mory-data-grids-explained  www.gridgain.com/blog/fyi/cache -data-grid-database/  infinispan.org  infinispan-ko.blogspot.com      
  33. 33. Thank you! http://www.slideshare.net/jaehongc

×