Infinispan Data Grid Platform

1,770 views
1,543 views

Published on

JBUG 2013 10th Anniversary Conference

Published in: Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,770
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
48
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Infinispan Data Grid Platform

  1. 1. Data Grid Platform 인피니스팬 소개와 사용 사례 전 재 홍 / Jaehong Cheon 9 Nov 2013
  2. 2. Agenda     Data Grid Infinispan Case Study References
  3. 3. Data Grid
  4. 4. Data Grid  Distributed Cache with persistence – – – – – Performance Boost Dynamic provisioning Fast access to data (in memory) - optionally write-through Elasticity Fault tolerance  Data Grid – Evolution of distributed caches – Well-known pattern to boost data access performance and scalability – Clustered by nature
  5. 5. Cache vs. Data Grid  JSR 107 - Temporary Caching for the Java Platform – read, write, expiry, write-through, distributed-manner – JBoss Cache  JSR 347 - Data Grids for the Java Platform – query, consistency, map-reducing standard way – Infinispan
  6. 6. Infinispan
  7. 7. Infinispan        Distributed In-memory key/value Data Grid/ Cache org.infinispan.Cache Interface Distributed as Library and Server (from 5.3) High availability Elastic Manageable Open source DefaultCacheManager manager = new DefaultCacheManager(); // Cache<Integer, Ticket> cache = manager.getCache(); Cache<Integer, Ticket> cache = manager.getCache(“myCache”);
  8. 8. Architecture: Library Library Mode - standalone Infinispan App JVM JCP-107 Style Cache just cache with advantages: expiry, j2ee transaction
  9. 9. Architecture: Library (Clustered)  Use as library Library Mode - clustered Infinispan – More features – Richer APIs – Programmatic/ Declarative configuration – Extendable/ embeddable – Faster (API call) App JVM Infinispan App Cluster JVM Infinispan App JVM Application doesn’t know it’s on cluster
  10. 10. Architecture: Server Server Mode - clustered  Use as server Infinispan JVM App App Infinispan Cluster JVM Infinispan App JVM – Remote  Memcached, R EST, Hot Rod, WebSocket – Data tier shared by multi-apps – App doesn’t affe ct cluster – Non-java clients  C++, .NET, Rub y, Python, Java
  11. 11. Architecture: Durability Durability  Durability Infinispan JVM Cluster Infinispan JVM Infinispan JVM Infinispan JVM Cluster persistence – By replication – By persistence – By replication to other cluster (topology aware)
  12. 12. Infinispan: Key Features     Transactions Persistence Querying Map/Reduce
  13. 13. Clustering  Peer-to-Peer – No central master, no single point of failure, no single bottle neck  JGroups – Reliable multicast communication library, nodes discovery, sharing data, performing cluster scaling  Consistent Hash – Hash based data distribution – How it finds where data locates  Linear in nature: throughput, capacity  Cluster Mode
  14. 14. Cluster Mode: Replication(복제) Replication Mode cache.put(K,V) Cache on Server 2 K,V Cache on Server 1 K,V Cache on Server 3 K,V Cache on Server 4 K,V
  15. 15. Cluster Mode: Distribution(분산) Distribution Mode(numOwners=2) cache.put(K,V) Cache on Server 1 K,V cache.get(K,V) Cache on Server 2 K,V Cache on Server 3 Cache on Server 4
  16. 16. Cluster Mode: Invalidation(무효화) Invalidation Mode cache.put(K,V2) Cache on Server 1 K,V2 Cache on Server 2 K,V Cache on Server 3 Cache on Server 4 DB
  17. 17. Configuration: Declarative <global> <transport clusterName="OperationsCacheCluster"> <properties> <property name="configurationFile“ value="jgroups-tcp.xml" /> </properties> </transport> <globalJmxStatistics enabled="true" /> </global> <default> <clustering mode="replication"> <sync /> </clustering> </default> <namedCache name="secureLayerContextCache"> <eviction strategy="LIRS" maxEntries="2000" /> <expiration lifespan="600000" /> <loaders passivation="true" shared="false" preload="false"> <fileStore fetchPersistentState="true" purgerThreads="3" purgeSynchronously="true" ignoreModifications="false" purgeOnStartup="false" location="${java.io.tmpdir}"> <async /> </fileStore> </loaders> </namedCache>  Eviction(제거)  Expiration(만료) – on cache – on key
  18. 18. Configuration: Programmatic  Configuration Based on XML DefaultCacheManager manager = new DefaultCacheManager("infinispan-config.xml"); Configuration baseConf = manager.getDefaultCacheConfiguration(); Configuration config =new ConfigurationBuilder(). read(baseConf).expiration().lifespan(50000).build(); manager.defineConfiguration(programmaticCache, config); Cache<String, String> cache = manager.getCache("secureLayerContextCache");  Programmatic configuration DefaultCacheManager manager = new DefaultCacheManager(); Configuration config = new ConfigurationBuilder() .loaders() .shared(false).passivation(false).preload(false) .addCacheLoader() .cacheLoader(new JdbcStringBasedCacheStore()) .addProperty("connectionFactoryClass","org.infinispan.loaders.jdbc .connectionfactory.ManagedConnectionFactory") .addProperty("datasourceJndiLocation", "java:jboss/datasources/MySQLDS") .addProperty("userName", "root") .addProperty("password", "admin") .async().threadPoolSize(10).build(); manager.defineConfiguration(programmaticCache, config); Cache<String, String> cache = manager.getCache("secureLayerContextCache");
  19. 19. Listener  Listener on CacheManager – Node join/ leave, Cache start/ stop  Cache – CRUD, Eviction/ Passivation – Rehashing/ Transaction completion @Listener public class SimpleListener { @CacheEntryCreated public void dataAdded(CacheEntryCreatedEvent event) { if (event.isPre()) { System.out.println("Before creating the entry:" + event.getKey()); } else { System.out.println("After creating the entry:" + event.getKey()); } … } DefaultCacheManager manager = new DefaultCacheManager(); manager.addListener(listener); Cache<Integer, Ticket> cache = manager.getCache(); cache.addListener(listener);
  20. 20. Asynchronous APIs  put() and get() and remove() are synchronous – They wait for RPC and Locks (and maybe cache stores)  The asynchronous API returns NotifyingFuture – Events are fired on completion of the operation NotifyingFuture<String> future = c.removeAsync(key); future.attachListener(new FutureListener<String>() { @Override public void futureDone(Future<String> future) { try { future.get(); System.out.printf ("The entry stored under key %s has been removed.", key); } catch (ExecutionException e) { System.out.printf("Failed to remove %s!", key); } } });
  21. 21. Key Features: Persistence  Used for durability  Cache Store - Persistence Storage – File System, Cloud, Remote, JDBC, JPA, LevelDB, Cassandra, – HBase, MongoDB, BerkeleyDB, JDBM, REST      CacheLoader, CacheStore(CacheWriter from 6.0) Write-through, write-behind Passivation, activation Store chain Shared store
  22. 22. Persistence: Passivation/Activation  Passivation – write to persistence when evicted from memory (default)  Activation – read to memory and remove from persistence
  23. 23. Key Features: Transactons  JTA Transaction Support  Support MVCC (Multi-Versioned Concurrency Control)  Isolation Level – READ_COMMITTED (default) – REPEATABLE_READ  Locking Mode – Optimistic Lock (default) – Pessimistic Lock
  24. 24. Key Features: Query  JBoss Hibernate Search + Apache Lucene  Query on values  Index Directory – Lucene Directory: in-memory, file system, JDBC – Infinispan Directory  Distributed queries
  25. 25. Distributed Execution  Executes codes on distributed nodes  Through a standard JDK ExecutorService interface  Use DistributedCallable extends java.util.concurrent.Callable
  26. 26. Key Features: Map/Reduce  Based on Distributed Execution Framework  Mapper, Reducer, Collator, MapReduceTask public interface Mapper<KIn, VIn, KOut, VOut> extends Serializable { void map(KIn key, VIn value, Collector<KOut, VOut> collector); } public interface Reducer<KOut, VOut> extends Serializable { VOut reduce(KOut reducedKey, Iterator<VOut> iter); } public interface Callator<KOut, Vout, R> { R collate(Map<KOut, VOut>); }
  27. 27. Client
  28. 28. Monitoring/Management  Mbeans on CacheManager, Cache  RHQ (JON, JBoss Operations Network)
  29. 29. Spring Integration  Infinispan provider for Spring cache abstraction  infinispan-spring.jar <cache:annotation-driven cache-manager="operationCacheManager"/> <bean id="operationCacheManager" class="org.infinispan.spring.provider.SpringEmbeddedCacheManagerFactoryBean" p:configurationFileLocation="classpath:infinispan-config.xml" /> @Cacheable(value = "secureLayerContextCache", key="#contextId") public SecureLayerContext getSecureLayerContext(String contextId) { return null; } @CachePut(value = "secureLayerContextCache", key="#contextId") public SecureLayerContext setSecureLayerContext(String contextId, SecureLayerContext secureLayerContext) { return secureLayerContext; } @CacheEvict(value = "secureLayerContextCache", key="#contextId") public void removeSecureLayerContext(String contextId) { // Intentionally blank }
  30. 30. Infinispan on Jboss AS 7  Used for session clustering, Hibernate L2 cache  Application gets cache with JNDI name using @Resource  XML Configuration in server configuration file <cache-container name="web" aliases="standard-session-cache" default-cache="repl"> <transport lock-timeout="60000" /> <replicated-cache name="repl" mode="ASYNC" batching="true"> <file-store /> </replicated-cache> </cache-container>
  31. 31. JDG     Red Hat JBoss Data Grid Infinispan-based JON All the benefits of subscription, including Red Hat world class support and services
  32. 32. Radar Gun  Data grid and distributed cache benchmarking framework  Built to test Infinispan and other distributed data grid platforms  https://github.com/radargun/radargun
  33. 33. Case Study
  34. 34. Case Study: Session Clustering  Store session information into cache in Spring MVC Interceptor
  35. 35. Case Study: Session Clustering Store session information into cache in Spring Security Filter - SecurityContextRepository를 구현한 CacheSecurityContextRepository 작성 loadContext, saveContext를 오버라이드하여 인피니스팬 사용 Spring cache abstraction 사용
  36. 36. Use Cases: Storm Processing State Store Infinispan Data Grid
  37. 37. References www.acornpub.co.kr/book/infinispan infinispan.org blog.infinispan.org infinispan-ko.blogspot.com facebook.com/groups/infinispan red.ht/data-grid tedwon.com/display /dev/Infinispan+Data+Grid  cbcpascal.blogspot.kr       
  38. 38. jbugkorea.org

×