galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Infinispan:
New Kid on the NoSQL Block
Galder Zamarreño
Senior Engineer, Red Hat
14th October 2010, Lausanne JUG
galder@jb...
“There is a need for a viable cloudready data store. People need to
rethink the way they organize, store
and access data.”...
Who is Galder?
• R&D engineer (Red Hat Inc):
• Infinispan developer
• JBoss Cache developer
• Contributor and committer:
•...
Agenda

• Cloud computing and data storage
• And why you should care!
• Data grids and cloud storage
• Introducing Infinis...
Clouds are today!
• Clouds are happening
• *aaS
• You cannot escape them!
• Public: Amazon, Google, Rackspace, ...
• Priva...
Why are clouds popular?
• Piecemeal costs, perfect utilization
• Pay for what you use, no more!
• Massive economies of sca...
Why should I care?
• My favorite platform is still relevant
• Java, Java EE
• Python, Ruby, .NET,... whatever!
• My favori...
Data Storage
• Databases on clouds:
• not a match made in heaven!
• Traditional modes of data storage won't work
• Clouds ...
RDBMS on clouds:
your options
• Non-ephemeral storage
• Restrictive
• Highly specialized hardware
• E.g., a SAN for Oracle...
Another solution: Data Grids!
• Data grids are perfect for clouds
• Highly scalable
• No single point of failure
• Works w...
Data Grids and other vendors
• Data grids
• Amazon SimpleDB uses Dynamo
• Google BigTable
• Infinispan
• Many other commer...
In-Memory Data Grids - Speed!
• Low latency
• minimal disk lookup
• Memory 2 orders of magnitude faster than disk
• especi...
Introducing

galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Introducing Infinispan
• Scalable data grid platform
• open source - LGPL
• based on some JBoss Cache code ... but mostly ...
galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Infinispan != JBoss Cache 4
• New architecture
• Brand new data container design
• Cutting edge algorithms
• New, complete...
More scalable than JBC
• Internal structures more memory-efficient
• Data organised in Map-like dictionaries
• As opposed ...
“Borrowed” from JBoss Cache
• JTA transactions
• Replicated data structure
• Fine-grained replication
• Eviction, cache pe...
… and new features!
• Consistent hash based data distribution
• Much simpler Map API (JSR-107 compliant)
• Ability to be c...
… and new features!

• JOPR based GUI management console
• JPA-like API
• Distributed execution
• Map/reduce made easy!

g...
Data distribution
• Consistent hash based data distribution
• Locating entries very efficient
• No network calls, no need ...
JPA-like API, fine-grained
replication
• Successor to POJO Cache
• JPA-like interface: persist, find, remove...
• Will not...
Management
• Uses JOPR, a rich web-based GUI
• Simple WAR file
• Open Source (LGPL)
• Infinispan exposes data, operations ...
So why is Infinispan sexy?

galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Why is Infinispan sexy?
• Transparent horizontal scalability
• Elastic in both directions
• Fast, low latency data access
...
The path ahead

galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Roadmap
• Infinispan 4.0.0 Starobrno (Released Feb 2010)
• New Map API
• Async API
• Distributed cache mode
• Management t...
Roadmap
• Infinispan 4.1.0 Radegast (Released August 2010)
• Client/server
• Memcached protocol
• Hot Rod protocol
• Smart...
Roadmap
• Infinispan 4.2.0 Ursus
• Collocated nodes in DIST
• Cassandra based cache store
• Infinispan 5.0.0 Pagoa
• JPA-l...
To sum it up
• Clouds are becoming mainstream
• Need to think about challenges
• DBs and clouds pose many challenges
• Dat...
How can YOU participate?
• Download and try it out!
• Report bugs in code, even docs, wikis, etc.
• Suggest new features!
...
Demo

galder@jboss.org | twitter.com/galderz | zamarreno.com
Monday, October 18, 2010
Questions?
infinispan.org
blog.infinispan.org
twitter.com/infinispan
#infinispan

galder@jboss.org | twitter.com/galderz |...
Upcoming SlideShare
Loading in...5
×

Infinispan - Galder Zamarreno - October 2010

221

Published on

Infinispan - Galder Zamarreno - October 2010

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
221
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Infinispan - Galder Zamarreno - October 2010

  1. 1. galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  2. 2. Infinispan: New Kid on the NoSQL Block Galder Zamarreño Senior Engineer, Red Hat 14th October 2010, Lausanne JUG galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  3. 3. “There is a need for a viable cloudready data store. People need to rethink the way they organize, store and access data.” galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  4. 4. Who is Galder? • R&D engineer (Red Hat Inc): • Infinispan developer • JBoss Cache developer • Contributor and committer: • JBoss AS, Hibernate, JGroups, JBoss Portal,...etc • Blog: zamarreno.com • Twitter: @galderz galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  5. 5. Agenda • Cloud computing and data storage • And why you should care! • Data grids and cloud storage • Introducing Infinispan galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  6. 6. Clouds are today! • Clouds are happening • *aaS • You cannot escape them! • Public: Amazon, Google, Rackspace, ... • Private: Red Hat, Oracle, VMWare, ... • Clouds will become mainstream • Traditional data centers become marginalized galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  7. 7. Why are clouds popular? • Piecemeal costs, perfect utilization • Pay for what you use, no more! • Massive economies of scale • High availability = Implicit backups! • Very fast provisioning -> Elasticity • Familiar charging model, controllable costs • Operational expenditure versus capital expenditure galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  8. 8. Why should I care? • My favorite platform is still relevant • Java, Java EE • Python, Ruby, .NET,... whatever! • My favorite OS is still relevant: • Linux • Solaris, ...etc. galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  9. 9. Data Storage • Databases on clouds: • not a match made in heaven! • Traditional modes of data storage won't work • Clouds are inherently stateless, ephemeral • Cloud deployments should scale • ... but databases still are a bottleneck • … and single point of failure! galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  10. 10. RDBMS on clouds: your options • Non-ephemeral storage • Restrictive • Highly specialized hardware • E.g., a SAN for Oracle RAC, ExaLogic? • Hardly commodity hardware! • Native database clustering • Unreliable, expensive galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  11. 11. Another solution: Data Grids! • Data grids are perfect for clouds • Highly scalable • No single point of failure • Works with ephemeral cloud nodes • Very low latency galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  12. 12. Data Grids and other vendors • Data grids • Amazon SimpleDB uses Dynamo • Google BigTable • Infinispan • Many other commercial and OSS offerings galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  13. 13. In-Memory Data Grids - Speed! • Low latency • minimal disk lookup • Memory 2 orders of magnitude faster than disk • especially for frequently used data • Concurrency, hardware threads • Disk IO is always a concurrency bottleneck • Memory offers far greater concurrency galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  14. 14. Introducing galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  15. 15. Introducing Infinispan • Scalable data grid platform • open source - LGPL • based on some JBoss Cache code ... but mostly all-new • JBoss Cache... • ... is a clustered caching library • ... exposes a tree-structured API • Infinispan has a Map-like API - (JSR-107 JCACHE) • ... so, primarily key/value NoSQL galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  16. 16. galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  17. 17. Infinispan != JBoss Cache 4 • New architecture • Brand new data container design • Cutting edge algorithms • New, completely different, APIs • Not backward-compatible • Although an code-level compatibility layer is available • New expectations • Designed for a far wider scope of purpose galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  18. 18. More scalable than JBC • Internal structures more memory-efficient • Data organised in Map-like dictionaries • As opposed to a tree • Making better use of CAS • Minimizing synchronized blocks, mutexes • Highly precise and low overhead data eviction • Uses JBoss Marshalling • smaller payloads + poolable streams = faster RPC galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  19. 19. “Borrowed” from JBoss Cache • JTA transactions • Replicated data structure • Fine-grained replication • Eviction, cache persistence • Notifications and eventing API • JMX reporting and Query API • MVCC locking • Non-blocking state transfer techniques galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  20. 20. … and new features! • Consistent hash based data distribution • Much simpler Map API (JSR-107 compliant) • Ability to be consumed by non-JVM platforms • Client/server module • Memcached compatibility • HotRod - binary protocol supporting “smart clients” • Javascript access via Websocket server • REST API galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  21. 21. … and new features! • JOPR based GUI management console • JPA-like API • Distributed execution • Map/reduce made easy! galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  22. 22. Data distribution • Consistent hash based data distribution • Locating entries very efficient • No network calls, no need for metadata • Will allow us to scale to bigger clusters • Goal of efficient scaling to 1000’s of nodes • Lightweight, “L1” cache for efficient reads • On writes, “L1” gets invalidated • Dynamic rebalancing galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  23. 23. JPA-like API, fine-grained replication • Successor to POJO Cache • JPA-like interface: persist, find, remove... • Will not rely on AOP, javassist, etc • More robust and easier to use/debug • Familiar JPA-like interface • Easy migration from existing, “traditional” data stores! galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  24. 24. Management • Uses JOPR, a rich web-based GUI • Simple WAR file • Open Source (LGPL) • Infinispan exposes data, operations in JMX • Infinispan-JOPR plugin represents this graphically • Other plugins can be built for other tools • HP OpenView, Hyperic, etc. galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  25. 25. So why is Infinispan sexy? galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  26. 26. Why is Infinispan sexy? • Transparent horizontal scalability • Elastic in both directions • Fast, low latency data access • Ability to address a very large heap • Cloud-ready datastore • Not just for Java • Free and doesn't suck! galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  27. 27. The path ahead galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  28. 28. Roadmap • Infinispan 4.0.0 Starobrno (Released Feb 2010) • New Map API • Async API • Distributed cache mode • Management tooling • REST API • Hibernate 2nd level cache galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  29. 29. Roadmap • Infinispan 4.1.0 Radegast (Released August 2010) • Client/server • Memcached protocol • Hot Rod protocol • Smart clients using HotRod • Websocket server • Lucene Directory • LIRS adaptive, recency-based eviction galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  30. 30. Roadmap • Infinispan 4.2.0 Ursus • Collocated nodes in DIST • Cassandra based cache store • Infinispan 5.0.0 Pagoa • JPA-like API + fine-grained replication • Distributed executors • Map/reduce programming model galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  31. 31. To sum it up • Clouds are becoming mainstream • Need to think about challenges • DBs and clouds pose many challenges • Data grids offer a good alternative • Infinispan, a new open source data grid • Viable cloud data store but not just for clouds • removes bottlenecks, single points of failure in non-cloud architectures too galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  32. 32. How can YOU participate? • Download and try it out! • Report bugs in code, even docs, wikis, etc. • Suggest new features! • Test with your own use cases and tell us how you use it!! • Lend a hand with development • Open and democratic dev process • Helps prioritize features you want! • Several non-Red Hat core committers already! galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  33. 33. Demo galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  34. 34. Questions? infinispan.org blog.infinispan.org twitter.com/infinispan #infinispan galder@jboss.org | twitter.com/galderz | zamarreno.com Monday, October 18, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×