Data Grid Platform
인피니스팬 소개와 사용 사례

전 재 홍 / Jaehong Cheon
9 Nov 2013
Agenda





Data Grid
Infinispan
Case Study
References
Data Grid
Data Grid
 Distributed Cache with persistence
–
–
–
–
–

Performance Boost
Dynamic provisioning
Fast access to data (in m...
Cache vs. Data Grid
 JSR 107 - Temporary Caching for the Java Platform
– read, write, expiry, write-through, distributed-...
Infinispan
Infinispan








Distributed In-memory key/value Data Grid/ Cache
org.infinispan.Cache Interface
Distributed as L...
Architecture: Library
Library Mode - standalone

Infinispan

App

JVM

JCP-107 Style Cache
just cache with advantages: exp...
Architecture: Library (Clustered)
 Use as library

Library Mode - clustered
Infinispan

– More features
– Richer APIs
– P...
Architecture: Server
Server Mode - clustered

 Use as server

Infinispan

JVM
App

App

Infinispan

Cluster

JVM
Infinisp...
Architecture: Durability
Durability

 Durability
Infinispan
JVM

Cluster
Infinispan
JVM

Infinispan
JVM

Infinispan
JVM

...
Infinispan: Key Features





Transactions
Persistence
Querying
Map/Reduce
Clustering
 Peer-to-Peer
– No central master, no single point of failure, no single bottle
neck

 JGroups
– Reliable mul...
Cluster Mode: Replication(복제)
Replication Mode
cache.put(K,V)

Cache on
Server 2 K,V

Cache on
Server 1 K,V

Cache on
Serv...
Cluster Mode: Distribution(분산)
Distribution Mode(numOwners=2)
cache.put(K,V)

Cache on
Server 1 K,V

cache.get(K,V)

Cache...
Cluster Mode: Invalidation(무효화)
Invalidation Mode
cache.put(K,V2)

Cache on
Server 1 K,V2

Cache on
Server 2 K,V

Cache on...
Configuration: Declarative
<global>
<transport clusterName="OperationsCacheCluster">
<properties>
<property name="configur...
Configuration: Programmatic
 Configuration Based on XML
DefaultCacheManager manager = new DefaultCacheManager("infinispan...
Listener
 Listener on CacheManager
– Node join/ leave, Cache start/ stop

 Cache
– CRUD, Eviction/ Passivation
– Rehashi...
Asynchronous APIs
 put() and get() and remove() are synchronous
– They wait for RPC and Locks (and maybe cache stores)

...
Key Features: Persistence
 Used for durability
 Cache Store - Persistence Storage
– File System, Cloud, Remote, JDBC, JP...
Persistence: Passivation/Activation
 Passivation – write to persistence when evicted from
memory (default)
 Activation –...
Key Features: Transactons
 JTA Transaction Support
 Support MVCC (Multi-Versioned Concurrency Control)
 Isolation Level...
Key Features: Query
 JBoss Hibernate Search + Apache Lucene
 Query on values
 Index Directory
– Lucene Directory: in-me...
Distributed Execution
 Executes codes on distributed nodes
 Through a standard JDK ExecutorService interface
 Use Distr...
Key Features: Map/Reduce
 Based on Distributed Execution Framework
 Mapper, Reducer, Collator, MapReduceTask
public inte...
Client
Monitoring/Management
 Mbeans on CacheManager, Cache
 RHQ (JON, JBoss Operations Network)
Spring Integration
 Infinispan provider for Spring cache abstraction
 infinispan-spring.jar
<cache:annotation-driven cac...
Infinispan on Jboss AS 7
 Used for session clustering, Hibernate L2 cache
 Application gets cache with JNDI name using
@...
JDG





Red Hat JBoss Data Grid
Infinispan-based
JON
All the benefits of
subscription, including
Red Hat world class
...
Radar Gun
 Data grid and distributed cache benchmarking
framework
 Built to test Infinispan and other distributed data g...
Case Study
Case Study: Session Clustering

 Store session information into cache
in Spring MVC Interceptor
Case Study: Session Clustering
Store session information into cache
in Spring Security Filter
-

-

SecurityContextReposit...
Use Cases: Storm Processing State Store

Infinispan Data Grid
References
www.acornpub.co.kr/book/infinispan
infinispan.org
blog.infinispan.org
infinispan-ko.blogspot.com
facebook.com/g...
jbugkorea.org
인피니스팬 데이터그리드 플랫폼
Upcoming SlideShare
Loading in …5
×

인피니스팬 데이터그리드 플랫폼

880 views

Published on

Infinispan Data Grid Platform
JBUG 2013 10th Anniversary Conference, 9 Nov 2013

Published in: Technology, Education

인피니스팬 데이터그리드 플랫폼

  1. 1. Data Grid Platform 인피니스팬 소개와 사용 사례 전 재 홍 / Jaehong Cheon 9 Nov 2013
  2. 2. Agenda     Data Grid Infinispan Case Study References
  3. 3. Data Grid
  4. 4. Data Grid  Distributed Cache with persistence – – – – – Performance Boost Dynamic provisioning Fast access to data (in memory) - optionally write-through Elasticity Fault tolerance  Data Grid – Evolution of distributed caches – Well-known pattern to boost data access performance and scalability – Clustered by nature
  5. 5. Cache vs. Data Grid  JSR 107 - Temporary Caching for the Java Platform – read, write, expiry, write-through, distributed-manner – JBoss Cache  JSR 347 - Data Grids for the Java Platform – query, consistency, map-reducing standard way – Infinispan
  6. 6. Infinispan
  7. 7. Infinispan        Distributed In-memory key/value Data Grid/ Cache org.infinispan.Cache Interface Distributed as Library and Server (from 5.3) High availability Elastic Manageable Open source DefaultCacheManager manager = new DefaultCacheManager(); // Cache<Integer, Ticket> cache = manager.getCache(); Cache<Integer, Ticket> cache = manager.getCache(“myCache”);
  8. 8. Architecture: Library Library Mode - standalone Infinispan App JVM JCP-107 Style Cache just cache with advantages: expiry, j2ee transaction
  9. 9. Architecture: Library (Clustered)  Use as library Library Mode - clustered Infinispan – More features – Richer APIs – Programmatic/ Declarative configuration – Extendable/ embeddable – Faster (API call) App JVM Infinispan App Cluster JVM Infinispan App JVM Application doesn’t know it’s on cluster
  10. 10. Architecture: Server Server Mode - clustered  Use as server Infinispan JVM App App Infinispan Cluster JVM Infinispan App JVM – Remote  Memcached, R EST, Hot Rod, WebSocket – Data tier shared by multi-apps – App doesn’t affe ct cluster – Non-java clients  C++, .NET, Rub y, Python, Java
  11. 11. Architecture: Durability Durability  Durability Infinispan JVM Cluster Infinispan JVM Infinispan JVM Infinispan JVM Cluster persistence – By replication – By persistence – By replication to other cluster (topology aware)
  12. 12. Infinispan: Key Features     Transactions Persistence Querying Map/Reduce
  13. 13. Clustering  Peer-to-Peer – No central master, no single point of failure, no single bottle neck  JGroups – Reliable multicast communication library, nodes discovery, sharing data, performing cluster scaling  Consistent Hash – Hash based data distribution – How it finds where data locates  Linear in nature: throughput, capacity  Cluster Mode
  14. 14. Cluster Mode: Replication(복제) Replication Mode cache.put(K,V) Cache on Server 2 K,V Cache on Server 1 K,V Cache on Server 3 K,V Cache on Server 4 K,V
  15. 15. Cluster Mode: Distribution(분산) Distribution Mode(numOwners=2) cache.put(K,V) Cache on Server 1 K,V cache.get(K,V) Cache on Server 2 K,V Cache on Server 3 Cache on Server 4
  16. 16. Cluster Mode: Invalidation(무효화) Invalidation Mode cache.put(K,V2) Cache on Server 1 K,V2 Cache on Server 2 K,V Cache on Server 3 Cache on Server 4 DB
  17. 17. Configuration: Declarative <global> <transport clusterName="OperationsCacheCluster"> <properties> <property name="configurationFile“ value="jgroups-tcp.xml" /> </properties> </transport> <globalJmxStatistics enabled="true" /> </global> <default> <clustering mode="replication"> <sync /> </clustering> </default> <namedCache name="secureLayerContextCache"> <eviction strategy="LIRS" maxEntries="2000" /> <expiration lifespan="600000" /> <loaders passivation="true" shared="false" preload="false"> <fileStore fetchPersistentState="true" purgerThreads="3" purgeSynchronously="true" ignoreModifications="false" purgeOnStartup="false" location="${java.io.tmpdir}"> <async /> </fileStore> </loaders> </namedCache>  Eviction(제거)  Expiration(만료) – on cache – on key
  18. 18. Configuration: Programmatic  Configuration Based on XML DefaultCacheManager manager = new DefaultCacheManager("infinispan-config.xml"); Configuration baseConf = manager.getDefaultCacheConfiguration(); Configuration config =new ConfigurationBuilder(). read(baseConf).expiration().lifespan(50000).build(); manager.defineConfiguration(programmaticCache, config); Cache<String, String> cache = manager.getCache("secureLayerContextCache");  Programmatic configuration DefaultCacheManager manager = new DefaultCacheManager(); Configuration config = new ConfigurationBuilder() .loaders() .shared(false).passivation(false).preload(false) .addCacheLoader() .cacheLoader(new JdbcStringBasedCacheStore()) .addProperty("connectionFactoryClass","org.infinispan.loaders.jdbc .connectionfactory.ManagedConnectionFactory") .addProperty("datasourceJndiLocation", "java:jboss/datasources/MySQLDS") .addProperty("userName", "root") .addProperty("password", "admin") .async().threadPoolSize(10).build(); manager.defineConfiguration(programmaticCache, config); Cache<String, String> cache = manager.getCache("secureLayerContextCache");
  19. 19. Listener  Listener on CacheManager – Node join/ leave, Cache start/ stop  Cache – CRUD, Eviction/ Passivation – Rehashing/ Transaction completion @Listener public class SimpleListener { @CacheEntryCreated public void dataAdded(CacheEntryCreatedEvent event) { if (event.isPre()) { System.out.println("Before creating the entry:" + event.getKey()); } else { System.out.println("After creating the entry:" + event.getKey()); } … } DefaultCacheManager manager = new DefaultCacheManager(); manager.addListener(listener); Cache<Integer, Ticket> cache = manager.getCache(); cache.addListener(listener);
  20. 20. Asynchronous APIs  put() and get() and remove() are synchronous – They wait for RPC and Locks (and maybe cache stores)  The asynchronous API returns NotifyingFuture – Events are fired on completion of the operation NotifyingFuture<String> future = c.removeAsync(key); future.attachListener(new FutureListener<String>() { @Override public void futureDone(Future<String> future) { try { future.get(); System.out.printf ("The entry stored under key %s has been removed.", key); } catch (ExecutionException e) { System.out.printf("Failed to remove %s!", key); } } });
  21. 21. Key Features: Persistence  Used for durability  Cache Store - Persistence Storage – File System, Cloud, Remote, JDBC, JPA, LevelDB, Cassandra, – HBase, MongoDB, BerkeleyDB, JDBM, REST      CacheLoader, CacheStore(CacheWriter from 6.0) Write-through, write-behind Passivation, activation Store chain Shared store
  22. 22. Persistence: Passivation/Activation  Passivation – write to persistence when evicted from memory (default)  Activation – read to memory and remove from persistence
  23. 23. Key Features: Transactons  JTA Transaction Support  Support MVCC (Multi-Versioned Concurrency Control)  Isolation Level – READ_COMMITTED (default) – REPEATABLE_READ  Locking Mode – Optimistic Lock (default) – Pessimistic Lock
  24. 24. Key Features: Query  JBoss Hibernate Search + Apache Lucene  Query on values  Index Directory – Lucene Directory: in-memory, file system, JDBC – Infinispan Directory  Distributed queries
  25. 25. Distributed Execution  Executes codes on distributed nodes  Through a standard JDK ExecutorService interface  Use DistributedCallable extends java.util.concurrent.Callable
  26. 26. Key Features: Map/Reduce  Based on Distributed Execution Framework  Mapper, Reducer, Collator, MapReduceTask public interface Mapper<KIn, VIn, KOut, VOut> extends Serializable { void map(KIn key, VIn value, Collector<KOut, VOut> collector); } public interface Reducer<KOut, VOut> extends Serializable { VOut reduce(KOut reducedKey, Iterator<VOut> iter); } public interface Callator<KOut, Vout, R> { R collate(Map<KOut, VOut>); }
  27. 27. Client
  28. 28. Monitoring/Management  Mbeans on CacheManager, Cache  RHQ (JON, JBoss Operations Network)
  29. 29. Spring Integration  Infinispan provider for Spring cache abstraction  infinispan-spring.jar <cache:annotation-driven cache-manager="operationCacheManager"/> <bean id="operationCacheManager" class="org.infinispan.spring.provider.SpringEmbeddedCacheManagerFactoryBean" p:configurationFileLocation="classpath:infinispan-config.xml" /> @Cacheable(value = "secureLayerContextCache", key="#contextId") public SecureLayerContext getSecureLayerContext(String contextId) { return null; } @CachePut(value = "secureLayerContextCache", key="#contextId") public SecureLayerContext setSecureLayerContext(String contextId, SecureLayerContext secureLayerContext) { return secureLayerContext; } @CacheEvict(value = "secureLayerContextCache", key="#contextId") public void removeSecureLayerContext(String contextId) { // Intentionally blank }
  30. 30. Infinispan on Jboss AS 7  Used for session clustering, Hibernate L2 cache  Application gets cache with JNDI name using @Resource  XML Configuration in server configuration file <cache-container name="web" aliases="standard-session-cache" default-cache="repl"> <transport lock-timeout="60000" /> <replicated-cache name="repl" mode="ASYNC" batching="true"> <file-store /> </replicated-cache> </cache-container>
  31. 31. JDG     Red Hat JBoss Data Grid Infinispan-based JON All the benefits of subscription, including Red Hat world class support and services
  32. 32. Radar Gun  Data grid and distributed cache benchmarking framework  Built to test Infinispan and other distributed data grid platforms  https://github.com/radargun/radargun
  33. 33. Case Study
  34. 34. Case Study: Session Clustering  Store session information into cache in Spring MVC Interceptor
  35. 35. Case Study: Session Clustering Store session information into cache in Spring Security Filter - - SecurityContextRepository를 구현한 CacheSecurityContextRepository 작성 loadContext, saveContext를 오버라이드하여 인피니스팬 사용 Spring cache abstraction 사용
  36. 36. Use Cases: Storm Processing State Store Infinispan Data Grid
  37. 37. References www.acornpub.co.kr/book/infinispan infinispan.org blog.infinispan.org infinispan-ko.blogspot.com facebook.com/groups/infinispan red.ht/data-grid tedwon.com/display /dev/Infinispan+Data+Grid  cbcpascal.blogspot.kr       
  38. 38. jbugkorea.org

×