Your SlideShare is downloading. ×
0
manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Infinispan    In-memory data grid meets NoSQLManik SurtaniFounder and project lead, InfinispanSenior Principal EngineerRed...
“Could data storage be the onething that hampers true cloudscalability and elasticity?”        manik@jboss.org | http://tw...
Who is Manik?• R&D Engineer, Red Hat Inc.   • Founder and project lead, Infinispan   • Project lead, JBoss Cache   • Frequ...
Agenda• What is Data-as-a-Service?• Introducing Infinispan• Implementing Data-as-a-Service with data grids• Data grids vs....
Traditional 3-tier App   manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Typical IaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Typical PaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Where’s yourdata stored??    manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Clouds areephemeral!    manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Statemanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Virtualizing Data• Some public services do exist • Amazon RDS and SimpleDB • FathomDB • Cloundant • MongoHQ • etc.        ...
What about private clouds?• Not all cloud deployments  are public!• Private cloud is very important• How can you build Daa...
Characteristics of DaaS• Elastic data• Needs to scale with other tiers• Response times should be linear• Needs to be highl...
“Traditional” RDBMSs• Lack of distribution hampers elasticity and HA• These limitations can be worked around • ... but thi...
“Traditional” RDBMSs• Lack of distribution hampers elasticity and HA• These limitations can be worked around • ... but thi...
Distributed Data Grid• Far better suited to elastic data• Distributed by nature• Highly available by nature• A good buildi...
API is king• Apps should use their native data storage APIs • E.g., JPA (Java EE), ActiveRecord (Ruby), etc.• Key/value to...
Introducingmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
What is Infinispan?• Open source (LGPL) in-memory Data Grid• Some concepts from Amazon Dynamo2 usage modes• Embedded• Clie...
Embedded Mode   manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
P2P Embedded Architecture      manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
API• Map-like key/value store• Upcoming JPA-like layer• Other high-level APIs being discussed in the  community e.g., Acti...
Consistent hash based distribution• Self healing• No single point of failureHighly concurrent• MVCC locking               ...
Persistence• Not just in memory!• Write through and write behind• Pluggable “drivers”Eviction and expiry• Efficient, adapt...
XA Transactions• 2-phase commit based• Deadlock detection algorithms• Coming soon: Atomic Broadcast       manik@jboss.org ...
Map/Reduce • In a pre-release state right now • Please try out 5.0.0.ALPHA3 with these APIs!Querying • Using Lucene and Hi...
Client/Server Mode    manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Client/Server Architecture                                        Supported Protocols                                     ...
WTF is Hot Rod?• Wire protocol for client server  communications• Open• Language independent• Built-in failover and load b...
Server Endpoint Comparison                           Client       Clustered Smart              Load Balancing/            ...
So is Infinispan a data grid?• In-memory• P2P, distributed• Low-latency, fast key/  value store                       mani...
So is Infinispan a data grid?• In-memory• P2P, distributed          ... or is it a NoSQL• Low-latency, fast key/  value st...
So is Infinispan a data grid?• In-memory• P2P, distributed           ... or is it a NoSQL• Low-latency, fast key/  value s...
DaaS withInfinispan    manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Architecture                                                                Manage and Monitormanik@jboss.org | http://twi...
Data Grids   vs. NoSQLmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
Data Grids vs. NoSQL• Data grids/distributed caches as proto-NoSQL?• Been around since early 2000’s• Used as database offl...
Their rootsData Gridsin-memory, non-clustered -> add clustering ->add persistence -> add txns -> add M/RNoSQLpersistence -...
Evolving: Enterprise NoSQL• Taking NoSQL mainstream• Adding enterprise features  • Transactions  • Querying• Adding common...
NoSQL standards• APIs - basic interactions and Map/Reduce  • JCP for Java + others    • http://bit.ly/data_grid_jsr• Wire ...
Summing things up• Elastic data is hard• Public data services not always suitable• Data grids make elastic storage easy• I...
Questions?    http://www.infinispan.org    http://blog.infinispan.org    http://github.com/infinispan    http://twitter.co...
Starting an Infinispan Server• Hot Rod or memcached server endpoint $ bin/startServer.sh -r           hotrod              ...
Starting an Infinispan Server• REST endpoint   • Deploy infinispan-server-rest.war in your     favorite servlet container....
Roadmap4.0.0 Starobrno• Map-like API• Async API• Consistent Hash based distribution• Write-through, write-behind• Eviction...
Roadmap4.1.0 Radegast• Deadlock detection• Client/Server protocols  • Memcached  • Hot Rod  • Smart clients using Hot Rod•...
Roadmap5.0.0 Pagoa• JPA-like API• Fine-grained replication• Distributed code execution  • Map/reduce• Virtual nodes for mo...
Upcoming SlideShare
Loading in...5
×

Infinspan: In-memory data grid meets NoSQL

6,115

Published on

Presented to the NoSQL User Group in Paris, France on the 7th of March, 2011

Published in: Technology

Transcript of "Infinspan: In-memory data grid meets NoSQL"

  1. 1. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  2. 2. Infinispan In-memory data grid meets NoSQLManik SurtaniFounder and project lead, InfinispanSenior Principal EngineerRed Hat, Inc. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  3. 3. “Could data storage be the onething that hampers true cloudscalability and elasticity?” manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  4. 4. Who is Manik?• R&D Engineer, Red Hat Inc. • Founder and project lead, Infinispan • Project lead, JBoss Cache • Frequent speaker on cloud computing and cloud data storage http://twitter.com/maniksurtani http://blog.infinispan.org manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  5. 5. Agenda• What is Data-as-a-Service?• Introducing Infinispan• Implementing Data-as-a-Service with data grids• Data grids vs. NoSQL manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  6. 6. Traditional 3-tier App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  7. 7. Typical IaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  8. 8. Typical PaaS App manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  9. 9. Where’s yourdata stored?? manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  10. 10. Clouds areephemeral! manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  11. 11. Statemanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  12. 12. Virtualizing Data• Some public services do exist • Amazon RDS and SimpleDB • FathomDB • Cloundant • MongoHQ • etc. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  13. 13. What about private clouds?• Not all cloud deployments are public!• Private cloud is very important• How can you build DaaS yourself? manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  14. 14. Characteristics of DaaS• Elastic data• Needs to scale with other tiers• Response times should be linear• Needs to be highly available • Nodes will die! The service shouldn’t. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  15. 15. “Traditional” RDBMSs• Lack of distribution hampers elasticity and HA• These limitations can be worked around • ... but this isn’t trivial • ... or cheap manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  16. 16. “Traditional” RDBMSs• Lack of distribution hampers elasticity and HA• These limitations can be worked around • ... but this isn’t trivial • ... or cheap manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  17. 17. Distributed Data Grid• Far better suited to elastic data• Distributed by nature• Highly available by nature• A good building block for your data service manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  18. 18. API is king• Apps should use their native data storage APIs • E.g., JPA (Java EE), ActiveRecord (Ruby), etc.• Key/value too low level • Akin to direct JDBC calls! manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  19. 19. Introducingmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  20. 20. What is Infinispan?• Open source (LGPL) in-memory Data Grid• Some concepts from Amazon Dynamo2 usage modes• Embedded• Client-server • memcached • Hot Rod • REST manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  21. 21. Embedded Mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  22. 22. P2P Embedded Architecture manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  23. 23. API• Map-like key/value store• Upcoming JPA-like layer• Other high-level APIs being discussed in the community e.g., ActiveRecord manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  24. 24. Consistent hash based distribution• Self healing• No single point of failureHighly concurrent• MVCC locking manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  25. 25. Persistence• Not just in memory!• Write through and write behind• Pluggable “drivers”Eviction and expiry• Efficient, adaptive algorithms• Addresses shortcomings of LRU & FIFO manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  26. 26. XA Transactions• 2-phase commit based• Deadlock detection algorithms• Coming soon: Atomic Broadcast manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  27. 27. Map/Reduce • In a pre-release state right now • Please try out 5.0.0.ALPHA3 with these APIs!Querying • Using Lucene and Hibernate Search to index manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  28. 28. Client/Server Mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  29. 29. Client/Server Architecture Supported Protocols • REST • Memcached • Hot Rod manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  30. 30. WTF is Hot Rod?• Wire protocol for client server communications• Open• Language independent• Built-in failover and load balancing• Smart routing manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  31. 31. Server Endpoint Comparison Client Clustered Smart Load Balancing/ Protocol Libraries ? Routing Failover REST Text N/A Yes No Any HTTP load balancer Only with predefinedMemcached Text Plenty Yes No server list Currently Hot Rod Binary Yes Yes Dynamic only Java manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  32. 32. So is Infinispan a data grid?• In-memory• P2P, distributed• Low-latency, fast key/ value store manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  33. 33. So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/ value store database? • Persistence • Map/Reduce • Client/Server mode manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  34. 34. So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/ value store database? • Persistence ... or something • Map/Reduce • Client/Server mode else? • Querying support • Transactional manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  35. 35. DaaS withInfinispan manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  36. 36. Architecture Manage and Monitormanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  37. 37. Data Grids vs. NoSQLmanik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  38. 38. Data Grids vs. NoSQL• Data grids/distributed caches as proto-NoSQL?• Been around since early 2000’s• Used as database offload • Bottleneck removal• Primary K/V stores for certain types of data• Low latency data access manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  39. 39. Their rootsData Gridsin-memory, non-clustered -> add clustering ->add persistence -> add txns -> add M/RNoSQLpersistence -> add clustering -> add M/R -> addmemory cache manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  40. 40. Evolving: Enterprise NoSQL• Taking NoSQL mainstream• Adding enterprise features • Transactions • Querying• Adding common APIs • Easy access, portability manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  41. 41. NoSQL standards• APIs - basic interactions and Map/Reduce • JCP for Java + others • http://bit.ly/data_grid_jsr• Wire protocol • Remote communications. • Hot Rod?• Query language • Without relational presumptions manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  42. 42. Summing things up• Elastic data is hard• Public data services not always suitable• Data grids make elastic storage easy• Infinispan server endpoints help build elastic data tiers• Discussed data grids versus NoSQL manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  43. 43. Questions? http://www.infinispan.org http://blog.infinispan.org http://github.com/infinispan http://twitter.com/infinispan - #infinispan #infinispan on FreeNode (IRC) manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  44. 44. Starting an Infinispan Server• Hot Rod or memcached server endpoint $ bin/startServer.sh -r hotrod -c infinispan.xml $ bin/startServer.sh -r memcached -c infinispan.xml manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  45. 45. Starting an Infinispan Server• REST endpoint • Deploy infinispan-server-rest.war in your favorite servlet container. manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  46. 46. Roadmap4.0.0 Starobrno• Map-like API• Async API• Consistent Hash based distribution• Write-through, write-behind• Eviction, expiration• Management tooling• REST API• Hibernate 2nd Level Cache• Released Feb 2010 manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  47. 47. Roadmap4.1.0 Radegast• Deadlock detection• Client/Server protocols • Memcached • Hot Rod • Smart clients using Hot Rod• Lucene Directory implementation• LIRS: adaptive, recency-based eviction policies• Released August 2010 manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  48. 48. Roadmap5.0.0 Pagoa• JPA-like API• Fine-grained replication• Distributed code execution • Map/reduce• Virtual nodes for more even distribution• In active development5.1.0 and beyond• Dynamic provisioning based on SLAs• Complex event processing features manik@jboss.org | http://twitter.com/maniksurtani | http://blog.infinispan.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×