Infinispan and Enterprise Data Grid
Upcoming SlideShare
Loading in...5
×
 

Infinispan and Enterprise Data Grid

on

  • 6,121 views

Tristan Tarrant - JBug Milano - January 2012

Tristan Tarrant - JBug Milano - January 2012

Statistics

Views

Total Views
6,121
Views on SlideShare
5,300
Embed Views
821

Actions

Likes
10
Downloads
146
Comments
0

14 Embeds 821

http://tedwon.com 674
http://www.extraordy.com 56
http://115.68.2.182 20
http://www.tedwon.com 17
http://extraordy-com.tumblr.com 12
http://tigerby.com 10
http://www.tumblr.com 9
http://www.party09.com 7
http://feeds.feedburner.com 6
http://www.tiger.com 4
http://play.daumcorp.com 3
http://from5.com 1
http://wiki.kthcorp.com 1
http://a0.twimg.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Infinispan and Enterprise Data Grid Infinispan and Enterprise Data Grid Presentation Transcript

  • Infinispan and Enterprise Data Grid Tristan Tarrant Principal Software Engineer [email_address] / https://twitter.com/tristantarrant/
  • Infinispan: What is it ?
    • In-memory key/value data grid
    • Built around java.util.ConcurrentMap interface
    • Easy to set-up and use
    Cache<String, String> cache = new DefaultCacheManager().getCache(); cache.put(“ash”, “nazg”); String v = cache.get(“ash”);
  • Infinispan: Clustering/1
    • Replication
      • All nodes contain a copy of every entry
      • Good for small clusters
      • Maximum size is constrained by node with least memory
    • Invalidation
      • Values are kept locally
      • Updates to a key invalidate all the other nodes
    • Distribution
      • Entries are placed on redundant nodes in the grid
      • Scales to larger clusters
      • Size scales linearly with the number of servers
  • Infinispan: Clustering/2
    • Consistent Hash
      • Calculated from the key
      • Location of data in the cluster is deterministic
      • Keeps RPCs low
    • L1 Cache
      • Remote data is proxied locally
      • Multiple retrievals of the same entry don't have to do RPC
    • Grouping
      • Uses “key group” to determine entry location
      • Keeps related keys together
  • Infinispan: Configuration
    • Programmatic via builder-style API
    • Declarative via XML
    CacheManager manager = new DefaultCacheManager(&quot;my-config-file.xml&quot;); Cache<String, String> myCache = manager.getCache(&quot;myCache&quot;); Configuration c1 = new ConfigurationBuilder() .clustering() .l1().disable() .mode(DIST_SYNC) .hash().numOwners(2) .build(); <infinispan> <global> <transport clusterName=&quot;infinispan-cluster /> </global> <namedCache name=&quot;myCache&quot;> <clustering mode=&quot;dist&quot;> <sync/> <hash numOwners=&quot;2&quot; /> <l1 enabled=&quot;false&quot;/> </clustering> </namedCache> </infinispan>
  • Infinispan: Asynchronous API
    • Normally put() and get() and remove() are synchronous
      • They wait for RPC and Locks (and maybe cache stores)
    • The asynchronous API returns NotifyingFutures for the above operations
      • Events are fired on completion of the operation
    FutureListener nineteenEightyFive = new FutureListener() { public void futureDone(Future future) { try { future.get(); } catch (Exception e) { System.out.println(&quot;The DeLorean is still in 1955&quot;); } } }; cache.putAsync(&quot;mph&quot;, &quot;88&quot;).attachListener(nineteenEightyFive);
  • Infinispan: Expiration & Eviction
    • Entries in the cache are immortal by default
    • Expiration (lifespan or idle time)
      • Per cache (via configuration)
      • Per key (programmatically)
    • Eviction
      • Maximum entry cap per cache
      • Various strategies (FIFO, LRU, LIRS)
    cache.put(“ash”, “nazg”, 5, SECONDS); cache.put(“krul”, “uruk”, 5, MINUTES, 1, MINUTE);
  • Infinispan: Listeners
    • Clients can register listeners for two classes of events
      • CacheManager
        • Node join/leave
        • Cache started/stopped
      • Cache
        • CRUD
        • Eviction / Passivation
        • Rehashing / Transaction completion
    • Listeners are annotated POJOs
    @Listener public class MyCacheListener { @CacheEntryModified public void print(CacheEntryModifiedEvent event) { System.out.println(“Something has changed in the Matrix”); } }
  • Infinispan: Transactions
    • Perform “atomic” operations on multiple keys
    • Caches are transactional or non-transactional (5.1)
    • MVCC
      • Readers don't acquire locks
      • Writers work on a copy until commit
    • Locking modes
      • Optimistic: locks acquired during prepare
      • Pessimistic: locks acquired during writes
    TransactionManager tm = cache.getAdvancedCache().getTransactionManager(); tm.begin(); cache.put(“ash”, “nazg”); cache.remove(“krul”); tm.commit();
  • Infinispan: Batching
    • Simpler and less expensive than transactions
    • Multiple operations are grouped and executed “atomically”
    • Integrates with JTA
    • Cannot participate in 2PC transactions
    cache.startBatch(); cache.put(&quot;ash&quot;, &quot;nazg&quot;); cache.put(&quot;krul&quot;, &quot;uruk&quot;); cache.put(&quot;gakh&quot;, &quot;olog&quot;); cache.endBatch(true); // commit cache.startBatch(); cache.put(&quot;krith&quot;, &quot;nazgu&quot;); cache.put(&quot;udu&quot;, &quot;gaathz&quot;); cache.endBatch(false); // rollback
  • Infinispan: Cache Stores
    • Persistent storage for the data in the grid
    • Pluggable stores
      • File, JDBC, BerkeleyDB, Cassandra, Remote
    • Support for passivation of evicted entries
    • Store chaining
    • Shared stores
  • Infinispan: Client Server
    • HotRod
      • Custom Infinispan protocol
      • Java, Python, .NET clients
      • Topology aware
    • Memcached
      • With all the features that memcached doesn't have :)
    • REST
      • Deployable WAR
      • Optimistic locking (via the ETag HTTP header)
  • Infinispan: Marshalling
    • Faster than standard Java serialization
    • Used for
      • Intra-node communication (replication, distribution)
      • Client-server communication (HotRod)
      • Persistence to a cache store
    • Support for custom marshallers
    • Support for non-serializable classes (via externalizer)
    • Lazy deserialization
      • Values are unmarshalled only when accessed
  • Infinispan: JGroups
    • Provides the communication trasport
    • Multiple protocol support
      • UDP
        • Multicast for automatic node discovery, datagrams for node to node, scales to many nodes
      • TCP
        • Static list of initial (stable), hosts, unicast for node to node, useful where multicast is disabled or across routers
      • Alternative discovery protocols (S3, JDBC, File, etc)
    • Advanced applications can use the underlying transport for their purposes (must use a MuxChannel)
    • Topology aware support (Site, Rack, Machine)
  • Infinispan: Query
    • Query on values
    • Uses Hibernate Search as Engine
    • Entry values are indexed via Lucene
    • Indexes may be
      • Local (each node keeps index of local entries, queries only return local data)
      • Replicated (all nodes have a copy of the full index, queries can be performed on the whole dataset)
      • Distributed (the index is stored in Infinispan, queries can be performed on the whole dataset)
    • Distributed queries (local index, query is executed on all nodes and aggregated)
  • Infinispan: Distributed Executors
    • A way to run code on all nodes
      • Modelled on Executor/Callable
    • Base for Map/Reduce
      • The entries in the cache are the input data
      • Map phase
        • A mapper method is called with each local entry as input
        • The mapper emits transformed keys/values to a collector
      • Reduce phase
        • A reducer method is called with all transformed values for every key
      • Collation phase
        • The transformed keys/values are aggregated from all nodes to the initiator
  • Infinispan: Monitoring
    • MBeans for CacheManagers and Caches
    • Statistics collection
      • Hits, Misses, Average R/W times
      • Transaction commits, rollbacks, deadlocks
      • Cluster members, replication timings
    • Operations
      • Cache start / stop
    • RHQ plugin
      • Monitor all metrics
      • Receive notifications on events
      • Aggregate information from multiple servers
  • Infinispan: JPA 2nd level cache
    • Query caching
      • Keyed by query-name and parameters hash
      • Cache can be local or replicated (only if it makes sense)
    • Entity caching
      • Keyed by the entity's primary key
      • Cache is local to each node
      • Invalidation is used to force other nodes to re-read from DB
    • Integrates with the JTA manager
      • Caches participate in transactions and support semantics
      • Strongly recommended
    • NB: Benchmark your application before enabling
  • Infinispan: CDI
    • Provides CDI support for
      • Cache configuration
      • Cache injection
      • JCache (JSR-107) annotations
      • Injection of caches from the AS7 Infinispan subsystem
    @Qualifier @Target({ElementType.FIELD, ElementType.PARAMETER, ElementType.METHOD}) @Retention(RetentionPolicy.RUNTIME) @Documented public @interface MyCache { } @ConfigureCache(&quot;myCache&quot;) @MyCache @Produces public Configuration myCacheConfiguration; @MyCache Cache<String,String> myCache;
  • Infinispan: Spring
    • Infinispan provider for Spring 3.1's cache abstraction
    • Integration with Spring's DI for constructing cache managers and caches
    • Declarative caching annotation
    • Declarative eviction annotation
    <bean id=&quot;cacheManager&quot; class=&quot;org.infinispan.spring.provider.SpringEmbeddedCacheManagerFactoryBean&quot; p:configurationFileLocation=&quot;classpath:infinispan-config.xml&quot; /> @Cacheable(&quot;items&quot;) public Item findItems(ItemPK itempk) {...} @CacheEvict(value = &quot;items&quot;, allEntries=true) public void readItems(ItemSource source) {...}
  • Infinispan: Continuous Query w/Drools
    • Distributed Complex Event Processing
    • Built on Drools Live Queries
    • Local and Clustered queries (replicated only for now)
    • Fault tolerant
    ContinuousQueryManager cqm = new ContinuousQueryManager(cacheManager); QueryDefinition qd = new QueryDefinition(queryName, query, outputEntries); cqm.defineQuery(qd); ContinuousQuery cq = cqm.executeContinuousQuery(qd.getQueryName(), true, params); cq.addQueryListener(new ResultSetListener() { public void entryAdded(MatchingEntry row) { // ... } public void entryRemoved(MatchingEntry row) { // ... } public void entryUpdated(MatchingEntry row) { // ... } });
  • Infinispan: 5.1 out TODAY!!!
    • Asymmetric clusters
    • Overhauled Transaction support
    • Single lock owner
    • Distributed Queries
    • Fine-Grained Atomic Maps
    • Uses JGroups 3.0 (with a lot of performance improvements)
    • New configuration based on builders
    • Many many performance optimizations and bugfixes
  • Infinispan: 5.2 ???
    • HotRod v2
      • Query
      • Map/Reduce (REST too)
      • Transactions
      • Events
    • Versioned entries
      • Will support eventual consistency
    • More work on optimizing the following areas:
      • Transactions
      • Locking
      • RPCs
  • JBoss AS7.x and Infinispan
    • AS7.x uses Infinispan for clustering
      • Web Sessions
      • Stateful Session EJB
      • JPA 2nd level cache
      • Session EJB failover
      • HA singleton
    • Configured via standalone.xml / domain.xml
      • Cache managers and caches injected via @Resource
      • User applications need to import the APIs
    META-INF/MANIFEST.MF Dependencies: org.infinispan export
  • Enterprise Data Grid: What is it ?
    • Productized Infinispan
    • Part of the Enterprise Platform 6 family of products
    • Three variants
      • Standalone (Infinispan Core)
      • Server (HotRod, Memcached, REST)
      • Embedded (Use of Infinispan within EAP 6)
  • Miscellanea
    • Come and help us !
      • Code http://github.com/infinispan/infinispan
      • Docs https://docs.jboss.org/author/display/ISPN/Home
      • Bug reporting https://issues.jboss.org/browse/ISPN
      • Community https://community.jboss.org/community/infinispan?view=discussions
    • Q & A
  • That's all folks [email_address] http://jboss.org/infinispan