Infinspan: In-memory data grid meets NoSQL

manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Infinispan In-memory data grid meets NoSQLManik SurtaniFounder and project lead, InfinispanSenior Principal EngineerRed Hat, Inc. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

“Could data storage be the onething that hampers true cloudscalability and elasticity?” manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Who is Manik?• R&D Engineer, Red Hat Inc. • Founder and project lead, Infinispan • Project lead, JBoss Cache • Frequent speaker on cloud computing and cloud data storage http://twitter.com/maniksurtani http://blog.infinispan.org manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Agenda• What is Data-as-a-Service?• Introducing Infinispan• Implementing Data-as-a-Service with data grids• Data grids vs. NoSQL manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Traditional 3-tier App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Typical IaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Typical PaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Where’s yourdata stored?? manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Clouds areephemeral! manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Statemanik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Virtualizing Data• Some public services do exist • Amazon RDS and SimpleDB • FathomDB • Cloundant • MongoHQ • etc. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

What about private clouds?• Not all cloud deployments are public!• Private cloud is very important• How can you build DaaS yourself? manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Characteristics of DaaS• Elastic data• Needs to scale with other tiers• Response times should be linear• Needs to be highly available • Nodes will die! The service shouldn’t. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

“Traditional” RDBMSs• Lack of distribution hampers elasticity and HA• These limitations can be worked around • ... but this isn’t trivial • ... or cheap manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Distributed Data Grid• Far better suited to elastic data• Distributed by nature• Highly available by nature• A good building block for your data service manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

API is king• Apps should use their native data storage APIs • E.g., JPA (Java EE), ActiveRecord (Ruby), etc.• Key/value too low level • Akin to direct JDBC calls! manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Introducingmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

What is Infinispan?• Open source (LGPL) in-memory Data Grid• Some concepts from Amazon Dynamo2 usage modes• Embedded• Client-server • memcached • Hot Rod • REST manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Embedded Mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

P2P Embedded Architecture manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

API• Map-like key/value store• Upcoming JPA-like layer• Other high-level APIs being discussed in the community e.g., ActiveRecord manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Consistent hash based distribution• Self healing• No single point of failureHighly concurrent• MVCC locking manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Persistence• Not just in memory!• Write through and write behind• Pluggable “drivers”Eviction and expiry• Efficient, adaptive algorithms• Addresses shortcomings of LRU & FIFO manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

XA Transactions• 2-phase commit based• Deadlock detection algorithms• Coming soon: Atomic Broadcast manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Map/Reduce • In a pre-release state right now • Please try out 5.0.0.ALPHA3 with these APIs!Querying • Using Lucene and Hibernate Search to index manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Client/Server Mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Client/Server Architecture Supported Protocols • REST • Memcached • Hot Rod manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

WTF is Hot Rod?• Wire protocol for client server communications• Open• Language independent• Built-in failover and load balancing• Smart routing manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Server Endpoint Comparison Client Clustered Smart Load Balancing/ Protocol Libraries ? Routing Failover REST Text N/A Yes No Any HTTP load balancer Only with predeﬁnedMemcached Text Plenty Yes No server list Currently Hot Rod Binary Yes Yes Dynamic only Java manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

So is Infinispan a data grid?• In-memory• P2P, distributed• Low-latency, fast key/ value store manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/ value store database? • Persistence • Map/Reduce • Client/Server mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/ value store database? • Persistence ... or something • Map/Reduce • Client/Server mode else? • Querying support • Transactional manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

DaaS withInfinispan manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Architecture Manage and Monitormanik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Data Grids vs. NoSQLmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Data Grids vs. NoSQL• Data grids/distributed caches as proto-NoSQL?• Been around since early 2000’s• Used as database offload • Bottleneck removal• Primary K/V stores for certain types of data• Low latency data access manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Their rootsData Gridsin-memory, non-clustered -> add clustering ->add persistence -> add txns -> add M/RNoSQLpersistence -> add clustering -> add M/R -> addmemory cache manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Evolving: Enterprise NoSQL• Taking NoSQL mainstream• Adding enterprise features • Transactions • Querying• Adding common APIs • Easy access, portability manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

NoSQL standards• APIs - basic interactions and Map/Reduce • JCP for Java + others • http://bit.ly/data_grid_jsr• Wire protocol • Remote communications. • Hot Rod?• Query language • Without relational presumptions manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Summing things up• Elastic data is hard• Public data services not always suitable• Data grids make elastic storage easy• Infinispan server endpoints help build elastic data tiers• Discussed data grids versus NoSQL manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Questions? http://www.infinispan.org http://blog.infinispan.org http://github.com/infinispan http://twitter.com/infinispan - #infinispan #infinispan on FreeNode (IRC) manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Starting an Infinispan Server• Hot Rod or memcached server endpoint $ bin/startServer.sh -r hotrod -c infinispan.xml $ bin/startServer.sh -r memcached -c infinispan.xml manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Starting an Infinispan Server• REST endpoint • Deploy infinispan-server-rest.war in your favorite servlet container. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Roadmap4.0.0 Starobrno• Map-like API• Async API• Consistent Hash based distribution• Write-through, write-behind• Eviction, expiration• Management tooling• REST API• Hibernate 2nd Level Cache• Released Feb 2010 manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Roadmap4.1.0 Radegast• Deadlock detection• Client/Server protocols • Memcached • Hot Rod • Smart clients using Hot Rod• Lucene Directory implementation• LIRS: adaptive, recency-based eviction policies• Released August 2010 manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Roadmap5.0.0 Pagoa• JPA-like API• Fine-grained replication• Distributed code execution • Map/reduce• Virtual nodes for more even distribution• In active development5.1.0 and beyond• Dynamic provisioning based on SLAs• Complex event processing features manik@jboss.org | http://twitter.com/maniksurtani | http://blog.inﬁnispan.org

Infinspan: In-memory data grid meets NoSQL

by Manik Surtani on Mar 08, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 8

Statistics

Infinspan: In-memory data grid meets NoSQL — Presentation Transcript