• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Hacking Infinispan: the new open source data grid meets NoSQL
 

Hacking Infinispan: the new open source data grid meets NoSQL

on

  • 2,713 views

La presentazione di Manik Surtani

La presentazione di Manik Surtani
in occasione del Codemotion, Roma 5 marzo 2011 http://www.codemotion.it

Statistics

Views

Total Views
2,713
Views on SlideShare
2,631
Embed Views
82

Actions

Likes
6
Downloads
0
Comments
0

4 Embeds 82

http://www.codemotion.it 64
http://2011.codemotion.it 12
http://codemotion.it 3
http://codemotion.macaronilab.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Hacking Infinispan: the new open source data grid meets NoSQL Hacking Infinispan: the new open source data grid meets NoSQL Presentation Transcript

    • Hacking Infinispan Open Source Data Grids meets NoSQL Manik Surtani Founder and Project Lead, Infinispan JBoss, by Red Hat Inc. Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • #infinispan Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • “Could data storage be the one thing that hampers true cloud scalability and elasticity?” Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Who is Manik? • R&D Engineer, Red Hat Inc. • Founder and project lead, Infinispan • Project lead, JBoss Cache • Frequent speaker on cloud computing and cloud data storage • http://twitter.com/maniksurtani • http://blog.infinispan.org Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • What is Data-as-a-Service? • Introducing Infinispan • Implementing Data-as-a-Service with Infinispan Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Traditional 3-tier App Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Typical IaaS App Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Typical PaaS App Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Where’s your data stored?? Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Clouds are ephemeral! Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Virtualizing Data • Some public services do exist • Amazon RDS and SimpleDB • FathomDB • Cloundant • MongoHQ • etc. Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • What about private clouds? • Not all cloud deployments are public! • Private cloud is very important • How can you build DaaS yourself? Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Characteristics of DaaS • Elastic data • Needs to scale with other tiers • Response times should be linear • Needs to be highly available • Nodes will die! The service shouldn’t. Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • SQL and NoSQL DB • Lack of distribution hampers elasticity and HA • These limitations can be worked around • ... but this isn’t trivial • ... or cheap Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • SQL and NoSQL DB • Lack of distribution hampers elasticity and HA • These limitations can be worked around • ... but this isn’t trivial • ... or cheap Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Distributed Data Grid • Far better suited to elastic data • Distributed by nature • Highly available by nature • A good building block for your data service Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • API is king • Apps should use their native data storage APIs • E.g., JPA (Java EE), ActiveRecord (Ruby), etc. • Key/value stores too low level • Akin to direct JDBC calls! Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Introducing Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • What is Infinispan? • Open source (LGPL) in-memory Data Grid • Written in Java and Scala • Some concepts from Amazon Dynamo • 2 usage modes • Embedded • Client-server • memcached, REST and Hot Rod Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Embedded Mode Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • P2P Embedded Architecture Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • API • Map-like key/value store • Upcoming JPA-like layer • Other high-level APIs being discussed in the community e.g., ActiveRecord Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • Consistent hash based distribution • Self healing • No single point of failure • Highly concurrent • MVCC locking Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • Persistence • Not just in memory! • Write through and write behind • Pluggable “drivers” • Eviction and expiry • Efficient, adaptive algorithms • Addresses shortcomings of LRU & FIFO Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • XA Transactions • 2-phase commit based • Deadlock detection algorithms • Coming soon: Atomic Broadcast Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • Map/Reduce • In a pre-release state right now • Please try out 5.0.0.ALPHA2 with these APIs! • Querying • Using Lucene and Hibernate Search to index Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Client/Server Mode Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Client/Server Architecture Supported Protocols •REST •Memcached •Hot Rod Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • WTF is Hot Rod? • Wire protocol for client server communications • Open • Language independent • Built-in failover and load balancing • Smart routing Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Server Endpoint Comparison Client Smart Load Balancing/ Protocol Clustered? Libraries Routing Failover Any HTTP load REST Text N/A Yes No balancer Only with predefined Memcached Text Plenty Yes No server list Currently Hot Rod Binary Yes Yes Dynamic only Java Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • So is Infinispan a data grid?• In-memory• P2P, distributed• Low-latency, fast key/value store Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/value database? store • Persistence • Map/Reduce Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • So is Infinispan a data grid?• In-memory• P2P, distributed ... or is it a NoSQL• Low-latency, fast key/value database? store • Persistence • Map/Reduce ... or something else? • Querying support • Transactional Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • So why is Infinispan sexy? Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • So why is Infinispan sexy? • Transparent horizontal scalability • Elastic in both directions • Fast, low latency data access • Ability to address a very large heap • Cloud-ready datastore • Not just for Java • Open Source Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • DaaS with Infinispan Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Architecture Manage and Monitor Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Where are we headed? Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • 4.0.0 Starobrno • Map-like API • Consistent Hash based distribution • Write-through, write-behind • Eviction, expiration • Management tooling • REST API • Hibernate 2nd Level Cache • Released Feb 2010 Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • 4.1.0 Radegast • Deadlock detection • Client/Server protocols • Memcached • Hot Rod • Smart clients using Hot Rod • Lucene Directory implementation • LIRS: adaptive, recency-based eviction policies • Released August 2010 Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • • 5.0.0 Pagoa • JPA-like API • Fine-grained replication • Distributed code execution • Map/reduce • Virtual nodes for more even distribution • In active development Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • What have we learned? • Elastic data is hard • Public data services not always suitable • Data grids make elastic storage easy • Infinispan server endpoints help build elastic data tiers Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011
    • Questions? • http://www.infinispan.org • http://blog.infinispan.org • http://github.com/infinispan • http://twitter.com/infinispan • #infinispan on FreeNode • http://bit.ly/rate_me_codemotion Manik Surtani http://twitter.com/maniksurtani http://blog.infinispan.orgSaturday, 5 March 2011