Infinispan – the open source data grid platform by Mircea Markus

•

1 like•1,269 views

This presentation describes what the project is and focuses on the main scenarios in which the audience can make use of it. Mircea Markus, project’s lead and co-founder, will give you an overview of the Infinispan ecosystem from which you’ll take home: - what Infinispan is - the main use cases in which you can benefit from it - its key features and differentiators in the data grid wold

Technology Education

Inﬁnispan - the open source
data grid platform
•Mircea Markus
Saturday, 23 March 13

Who’s this guy?
•JBoss clustering: JBossCache, JGroups
•Lead Inﬁnispan
•JSR-350
•@inﬁnispan/@mirceamarkus
•inﬁnispan.blogspot.com
Saturday, 23 March 13

Agenda
•What is Inﬁnispan
•API
•Key features
•4 use cases
Saturday, 23 March 13

What is Inﬁnispan?
•Distributed, in memory, data structure
•Highly available
•Elastic
•Open source
Saturday, 23 March 13

Distributed data structure
Storage Engine
(RAM
+Overﬂow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
Storage Engine
(RAM
+Overﬂow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
TCP/UDP
RemoteClient
TCP
TCP
Saturday, 23 March 13

High availability
•Memory is volatile
•Make redundant copies
•Total replication (Replication Mode)
•Partial replication (Distribution Mode)
•numOwners
•Topology changes
•Node will crash!
•Re-arrange state
Saturday, 23 March 13

Elasticity
•Expect
•Node additions
•Node removals
•Topology changes
•are totally consistent
•do not "stop the world"
Saturday, 23 March 13

API
•Key-value store
•java.util.concurrent.ConcurrentHashMap
•JSR-107 compliant
•CDI support
Saturday, 23 March 13

Key features
•Transactions
•Persistence
•Map/Reduce
•Querying
Saturday, 23 March 13

Transactions
•JTA transactions support
•XA or Synchronization based enlistment
•recovery!
•Local transactions (batching)
•Transactional modes
•optimistic
•pessimistic
Saturday, 23 March 13

Persistence/Cache Store
•Used for
•durability
•increased storage
•warm caches
•Various implementations
•jdbc, bdbje, ﬁle, cloud, remote/inﬁnispan, cassandra
•Extensible
Saturday, 23 March 13

Map/Reduce example
Saturday, 23 March 13

Querying
•Based on Hibernate Search/Lucene
•Objects in the grid are indexed
•index management
Saturday, 23 March 13

Use cases
•Local cache
•Cluster of caches
•Autonomous data store
•access protocols
•Geographical failover
Saturday, 23 March 13

Features of a local cache
•Eviction
•Expiry
•Write through, write behind
•Preloading
•Notiﬁcations
•Statistics
Saturday, 23 March 13

Local cache not good enough...
Saturday, 23 March 13

Some limitations still
•Client is affected by cache topology changes
•Shared resources
•Tier management
•incompatible JVM tuning
•security
•garbage collection
•Non-JVM clients
Saturday, 23 March 13

Client/Server protocols
•REST
•Memcached
•Hotrod
•proprietary
•java, C#, python, ruby
•others on the way..
Saturday, 23 March 13

Geographic failover
Inﬁnispan
Node
Inﬁnispan
Node
Inﬁnispan Cluster
Inﬁnispan
Node
London
Inﬁnispan
Node
Inﬁnispan
Node
Inﬁnispan Cluster
Inﬁnispan
Node
New York(Active)
Inﬁnispan
Node
Inﬁnispan
Node
Inﬁnispan Cluster
Inﬁnispan
Node
San Francisco
Migrate Data
Migrate Data
Migrate Data
Saturday, 23 March 13

JBoss Data Grid (JDG)
•JBoss Data Grid
•Build on top of Iniﬁnispan
•http://www.jboss.com/edg
Saturday, 23 March 13

Thank you!
•inﬁnispan.blogspot.com
•@inﬁnispan
•mirceamarkus.blogspot.com
•@mirceamarkus
Saturday, 23 March 13

What's hot

State of the_gluster_-_lceu

Gluster.org

NetFlow Data processing using Hadoop and Vertica

Josef Niedermeier

Presentation Summary: By leveraging on memory mapped files, the Chronicle Engine supports large maps that easily can exceed the size of your server’s RAM, thus allowing application developers to create huge JVM:s where data can be obtained quickly and with predictable latency. The Chronicle Engine can be synchronized with an underlying database using Speedment so that your in-memory maps will be “alive” and change whenever data changes in the underlying database. Speedment can also automatically derive domain models directly from the database so that you can start using the solution very quickly. Because the Java Maps are mapped onto files, the maps can also be shared instantly between several JVM:s and when you restart a JVM, it may start very quickly without having to reload data from the underlying database. The mapped files can be hundreds of terabytes which has been done in real world deployment cases.

Java one2015 - Work With Hundreds of Hot Terabytes in JVMs

Speedment, Inc.

Gluster.next feb-2016

Vijay Bellur

Cache options for Data Layer

Hussain Mansoor

NoSQL solutions

Felix Crisan

Web scale monitoring

Dobrica Pavlinušić

Gluster.community.day.2013

Udo Seidel

What's hot (8)

State of the_gluster_-_lceu

NetFlow Data processing using Hadoop and Vertica

Java one2015 - Work With Hundreds of Hot Terabytes in JVMs

Gluster.next feb-2016

Cache options for Data Layer

NoSQL solutions

Web scale monitoring

Gluster.community.day.2013

Viewers also liked

Infinispan,Lucene,Hibername OGM

JBug Italy

In this session, Galder Zamarreño, a senior software engineer at Red Hat, will: - Provide a brief introduction to RESTful principles Discuss how cloud-scale APIs are done best with REST - Introduce Infinispan REST server, focusing on its cloud capabilities and simple REST API - Detail how REST can apply to many APIs, focusing on some of the deeper principles and practices behind it and how easy it is to implement and use

Why RESTful Design for the Cloud is Best

Galder Zamarreño

Presentation delivered by Mircea Markus (JBoss) at the London JBoss User Group Event on Wednesday, the 4th of December 2013. In this talk, Mircea Markus (Infinispan Lead) covers the major features added in the 6.0 release to the Infinispan ecosystem: - querying in client/server mode - better integration with the persistent storage - multi-master cross-site replication - support for heterogeneous clusters Participants will take home a better understanding of Infinispan capabilities and the use cases in which it can be put at work. Ideally the attendees should have a basic knowledge of the in-memory data grids.

What's New in Infinispan 6.0

JBUG London

Infinispan

guest69f868

Infinispan Data Grid Platform

jbugkorea

London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server

JBUG London

Introducing Infinispan

PT.JUG

Data Grids vs Databases

Galder Zamarreño

Infinispan, a distributed in-memory key/value data grid and cache

Sebastian Andrasoni

Data Grids and Data Caching

Galder Zamarreño

Infinispan for Dummies

Galder Zamarreño

Viewers also liked (11)

Infinispan,Lucene,Hibername OGM

Why RESTful Design for the Cloud is Best

What's New in Infinispan 6.0

Infinispan

Infinispan Data Grid Platform

London JBUG April 2015 - Performance Tuning Apps with WildFly Application Server

Introducing Infinispan

Data Grids vs Databases

Infinispan, a distributed in-memory key/value data grid and cache

Data Grids and Data Caching

Infinispan for Dummies

Similar to Infinispan – the open source data grid platform by Mircea Markus

MongoDB Capacity Planning

Norberto Leite

Deploying any software can be a challenge if you don't understand how resources are used or how to plan for the capacity of your systems. Whether you need to deploy or grow a single MongoDB instance, replica set, or tens of sharded clusters then you probably share the same challenges in trying to size that deployment. This webinar will cover what resources MongoDB uses, and how to plan for their use in your deployment. Topics covered will include understanding how to model and plan capacity needs for new and growing deployments. The goal of this webinar will be to provide you with the tools needed to be successful in managing your MongoDB capacity planning tasks.

Webinar: Capacity Planning

MongoDB

pkgsrc on SmartOS

jonperkin

Hotspot Garbage Collection - Tuning Guide

jClarity

Cloud infrastructure. Google File System and MapReduce - Andrii Vozniuk

Andrii Vozniuk

Hands on Performance Tuning - Mike Croft

JAXLondon2014

Hands-on Performance Workshop - The science of performance

C2B2 Consulting

We have had the luxury of conducting security research and penetration testing activities on a few Supercomputers on the world top 500 list. Having spent some time assessing the security of these awesome machines it’s fair to say that many of the technologies involved have been largely untouched by public security scrutiny and consequently the security bar is much lower than we have come to expect in 2013. This presentation will cover our research and demonstrate some of the most interesting and significant vulnerabilities we have uncovered so far. We will also be demonstrating 0-day exploits and previously undocumented attack techniques live so you can see how to get root on 20,000 nodes all at once. There are many ways to hack the Gibson and today you’ll learn some.

44CON 2013 - Hack the Gibson - Exploiting Supercomputers - John Fitzpatrick &...

44CON

IBM Spark Technology Center: Real-time Advanced Analytics and Machine Learnin...

DataStax Academy

JAVA MICROSERVICES FOR BIG DATA WITH LOW LATENCY - Per-Ake Minborg, CTO Speedment By leveraging on memory mapped files (eg. Hazelcast, ChronicleMaps etc.), Speedment supports large Java Maps that easily can exceed the size of your server’s RAM. Because the Java Maps are mapped onto files, these maps can be shared instantly between several microservice JVMs and new microservice instances can be added, removed or restarted very quickly. Data can be retrieved with predictable ultra-low latency for a wide range of operations. The solution can be synchronized with an underlying database so that your in-memory maps will be consistently “alive”. The mapped files can be terabytes which has been done in real world deployment cases and there can be a large number of microservices that shares these maps simultaneously.

NYJavaSIG - Big Data Microservices w/ Speedment

Speedment, Inc.

MAGPI: Advanced Services: IPv6, Multicast, DNSSEC

Shumon Huque

Outsourced database

faraz Safarpour

Writing Scalable Software in Java

Ruben Badaró

Outsourced database

faraz Safarpour

Monitoring MongoDB’s Engines in the Wild

Tim Vaillancourt

SharkFest16_Palm_Online

Brad Palm

OracleHA.ppt

APras2

Pldc2012 monitoring-and-trending-with-mysql

radiocats

Lessons from database failures

Colin Charles

Yet Another Replication Tool: RubyRep

Denish Patel

Similar to Infinispan – the open source data grid platform by Mircea Markus (20)

MongoDB Capacity Planning

Webinar: Capacity Planning

pkgsrc on SmartOS

Hotspot Garbage Collection - Tuning Guide

Cloud infrastructure. Google File System and MapReduce - Andrii Vozniuk

Hands on Performance Tuning - Mike Croft

Hands-on Performance Workshop - The science of performance

44CON 2013 - Hack the Gibson - Exploiting Supercomputers - John Fitzpatrick &...

IBM Spark Technology Center: Real-time Advanced Analytics and Machine Learnin...

NYJavaSIG - Big Data Microservices w/ Speedment

MAGPI: Advanced Services: IPv6, Multicast, DNSSEC

Outsourced database

Writing Scalable Software in Java

Outsourced database

Monitoring MongoDB’s Engines in the Wild

SharkFest16_Palm_Online

OracleHA.ppt

Pldc2012 monitoring-and-trending-with-mysql

Lessons from database failures

Yet Another Replication Tool: RubyRep

More from Codemotion

Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...

Codemotion

Pompili - From hero to_zero: The FatalNoise neverending story

Codemotion

Pastore - Commodore 65 - La storia

Codemotion

Rivivere l'ebbrezza di progettare un vecchio computer o una consolle da bar è oggi possibile sfruttando le FPGA, ovvero logiche programmabili che consentono a chiunque di progettare il proprio hardware o di ricrearne uno del passato. In questa sessione si racconta come dal reverse engineering dell'hardware di vecchie glorie come il Commodore 64 e lo ZX Spectrum sia stato possibile farle rivivere attraverso tecnologie oggi alla portata di tutti.

Pennisi - Essere Richard Altwasser

Codemotion

There's a lot of talk about blockchain, but how does the technology behind it actually work? For developers, getting some hands-on experience is the fastest way to get familiair with new technologies. So let's build a blockchain, then! In this session, we're going to build one in plain old Java, and have it working in 40 minutes. We'll cover key concepts of a blockchain: transactions, blocks, mining, proof-of-work, and reaching consensus in the blockchain network. After this session, you'll have a better understanding of core aspects of blockchain technology.

Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...

Codemotion

When was the last time you were truly lost? Thanks to the maps and location technology in our phones, a whole generation has now grown up in a world where getting lost is truly a thing of the past. Location technology goes far beyond maps in the palm of our hand, however. In this talk, we will explore how a ridesharing app works. How do we discover our destination?How do we find the closest driver? How do we display this information on a map? How do we find the best route?To answer these questions,we will be learning about a variety of location APIs, including Maps, Positioning, Geocoding etc.

Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019

Codemotion

Eward Driehuis, SecureLink's research chief, will guide you through the bumpy ride we call the cyber threat landscape. As the industry has over a decade of experience of dealing with increasingly sophisticated attacks, you might be surprised to hear more attacks slip through the cracks than ever. From analyzing 20.000 of them in 2018, backed by a quarter of a million security events and over ten trillion data points, Eward will outline why this happens, how attacks are changing, and why it doesn't matter how neatly or securely you code.

Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019

Codemotion

IoT revolution is ended. Thanks to hardware improvement, building an intelligent ecosystem is easier than never before for both startups and large-scale enterprises. The real challenge is now to connect, process, store and analyze data: in the cloud, but also, at the edge. We’ll give a quick look on frameworks that aggregate dispersed devices data into a single global optimized system allowing to improve operational efficiency, to predict maintenance, to track asset in real-time, to secure cloud-connected devices and much more.

Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -

Codemotion

What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. A team from TNG Technology Consulting managed to build an AI showcase using OpenCV and Tensorflow to realize such goggles.

Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...

Codemotion

Blockchain (and Cryptocurrency) is an evolution of 20-year old research from scientists like Chaum, Lamport, and Castro & Liskov. Due to the current hype, it's hard to distinguish beneficial aspects of the technology from a desire for a "silver bullet" for device security, verifiable logistics, or "saving democracy". The problem: blockchain introduces new security challenges - and blind adoption without understanding reduces overall security. In this talk, Melanie Rieback and Klaus Kursawe explain the pitfalls and limits of blockchain, so you can avoid making your applications LESS secure.

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...

Codemotion

Networking is a core part of computing in the digital world we inhabit. But, how well do you know how it works? Do you understand all the moving parts of the OSI stack inside your computer, and how the network is actually put together? How can this ever work? This guided safari of layers, standards, protocols, and happenstance will bring us close to the copper wire, and up through the layers of CDMA/CD, ARP, routing and HTTP. We will make a few excursions through patchworks that still work forty years later, and cleverly designed mechanisms that show that simplicity is the only way to last.

Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...

Codemotion

Performance tests are not only an important instrument for understanding a system and its runtime environment. It is also essential in order to check stability and scalability – non-functional requirements that might be decisive for success. But won't my cloud hosting service scale for me as long as I can afford it? Yes, but… It only operates and scales resources. It won't automatically make your system fast, stable and scalable. This talk shows how such and comparable questions can be clarified with performance tests and how DevOps teams benefit from regular test practise.

Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...

Codemotion

Sascha will demonstrate the opportunities and challenges of Conversational AI learned from the practice. Both Technology and User Experience will be covered introducing a process finding micro-moments, writing happy paths, gathering intents, designing the conversational flow, and finally publishing on almost all channels including Voice Services and Chatbots. Valuable for enterprises, developers, and designers. All live on stage in just minutes and with almost no code.

Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019

Codemotion

A key challenge we face at Pacmed is quickly calibrating and deploying our tools for clinical decision support in different hospitals, where data formats may vary greatly. Using Intensive Care Units as a case study, I’ll delve into our scalable Python pipeline, which leverages Pandas’ split-apply-combine approach to perform complex feature engineering and automatic quality checks on large time-varying data, e.g. vital signs. I’ll show how we use the resulting flexible and interpretable dataframes to quickly (re)train our models to predict mortality, discharge, and medical complications.

Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019

Codemotion

Coolblue is a proud Dutch company, with a large internal development department; one that truly takes CI/CD to heart. Empowerment through automation is at the heart of these development teams, and with more than 1000 deployments a day, we think it's working out quite well. In this session, Pat Hermens (a Development Managers) will step you through what enables us to move so quickly, which tools we use, and most importantly, the mindset that is required to enable development teams to deliver at such a rapid pace.

Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019

Codemotion

Quantum computers can use all of the possible pathways generated by quantum decisions to solve problems that will forever remain intractable to classical compute power. As the mega players vie for quantum supremacy and Rigetti announces its $1M "quantum advantage" prize, we live in exciting times. IBM-Q and Microsoft Q# are two ways you can learn to program quantum computers so that you're ready when the quantum revolution comes. I'll demonstrate some quantum solutions to problems that will forever be out of reach of classical, including organic chemistry and large number factorisation.

James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...

Codemotion

Chinese food exploded across America in the early 20th century, rapidly adapting to local tastes while also spreading like wildfire. How was it able to spread so fast? The GY6 is a family of scooter engines that has achieved near total ubiquity in Europe. It is reliable and cheap to manufacture, and it's made in factories across China. How are these factories able to remain afloat? Chinese-American food and the GY6 are both riveting studies in product-market fit, and both are the product of a distributed open source-like development model. What lessons can we learn for open source software?

Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...

Codemotion

Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019

Codemotion

Would you fly in a plane designed by a craftsman or would you prefer your aircraft to be designed by engineers? We are learning that science and empiricism works in software development, maybe now is the time to redefine what “Software Engineering” really means. Software isn't bridge-building, it is not car or aircraft development either, but then neither is Chemical Engineering. Engineering is different in different disciplines. Maybe it is time for us to begin thinking about retrieving the term "Software Engineering" maybe it is time to define what our "Engineering" discipline should be.

Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019

Codemotion

What is the job of a CTO and how does it change as a startup grows in size and scale? As a CTO, where should you spend your focus? As an engineer aspiring to be a CTO, what skills should you pursue? In this inspiring and personal talk, I describe my journey from early Red Hat engineer to CTO at Bloomon. I will share my view on what it means to be a CTO, and ultimately answer the question: Should the CTO be coding?

Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019

Codemotion

More from Codemotion (20)