SlideShare a Scribd company logo
APACHE IGNITE
THE NEXT PHASE OF DISTRIBUTED
SYSTEMS WITH
OMG IT’S LIKE TOTALLY ME….
DANI TRAPHAGEN - @DTRAPEZOID
▸ The person in front of the room
▸ This doesn’t make me more special than
you
▸ This just means I spent an afternoon of my
life submitting an abstract
▸ No, but seriously, I did that
▸ But here’s a picture of me looking totes
professional
▸ Oh also, I used to consult on NoSQL
databases and now I don’t…because I sort
of question………….
IS MEMORY…
WELL WHAT ARE
THE TRENDS?
SOURCE HERE
SO THE TREND IS MEMORY IS
DECREASING IN COST.
COOL.
SO WHAT IN THE
H-E-
IS APACHE IGNITE?
APACHE IGNITE
A LITTLE HISTORY ON
A WEE BIT OF HISTORY
APACHE IGNITE’S HISTORY
▸ Apache Ignite hits the streets in Oct. 2014, released by
GridGain
▸ Aug. 2015 Ignite is the 2nd fastest project to graduate
after Spark
▸ Today, over 100 Contributors internationally & rapidly
growing out the community
▸ Over 1m lines of code
BASIC APACHE IGNITE
ARCHITECTURE
SPEEDS
UP
140 23 1
5067 7898 45
‣ HPC
‣ Machine
learning
‣ Risk analysis
‣ Grid computing
‣ HA API Services
‣ Scalable
Middleware
‣ Distributed
caching
‣ KeyValue Store
‣ In-Memory SQL
‣ APIs for Kafka,
etc.,as well as
Ignite’s
‣ Lambda
Architectures
‣ Big Data
‣ Realtime
Analytics
‣ Batch
processing
‣ Distributed In-
Memory File
System
‣ Node2Node &
Topic-based
Messaging
‣ Fault Tolerance
‣ Multiple
backups
‣ Cluster groups
‣ Auto
‣ CEP,Streaming
Windows &
Cont.Queries
‣ Event driven
design
‣ Dist.queues
‣ Atomic
variables
‣ Dist.
Semaphore
APACHE IGNITE IS MORE THAN A BASIC
CACHE, IT’S A PLATFORM
DATA GRID
This is the foundation of Ignite - rebalances the data in a
load balanced and fault tolerant fashion
KEY VALUE STORE & HIGH
AVAILABILITY WITH APACHE IGNITE
▸ Ignite’s Data Grid is a Distributed
Key Value Store, Caching
Distributed In-Memory Data
Throughout the Cluster
▸ Two Flavors of Cache: Partitioned
or Replicated
▸ Consistency
▸ Data Locality
SQL GRID
MICROSERVICES DEPLOYMENT WITH APACHE IGNITE
▸ The Service Grid is the foundation for a Microservices-
based Architecture
▸ Aspects of the Microservices Solution
▸ Data Nodes, Service Nodes
▸ Communication
▸ Internal and External Apps
▸ Persistence
SERVICE GRID
▸Load-
Balanced
▸Fault
Tolerant
▸ Deployment - Node Filter - a class that allows you to
specify nodes to run your service
▸ Service Grid APIs - Pass Service Service Proxy
▸ Load Balancing - Sticky vs. Non-Sticky
▸ Manageable Deployment - Not necessary everywhere
in your cluster, only where you need services
deployment
RECAP
▸ Manage the Lifecycle of your Services
▸ Initiate, Maintain, Kill
▸ Load Balancing
▸ Requests should be going to the services you have
▸ Fault Tolerance
▸ If a service goes down, it should be addressed immediately
▸ Scalability
▸ If volume demands change, you need to be able to handle it
Persistent Storage
EXTERNAL MICROSERVICES APPLICATIONS
DATA
NODE
DATA
NODE
DATA
NODE
DATA
NODE
Service Node Service Node Service Node
MS1MS1
MS2MS3MS2
DATA NODES -
▸ Server nodes that store the data & accepts queries (&
or computations) in a distributed fashion
▸ Provides distributed storage, don’t need provide data
model classes, services classes.
SERVICE NODES
▸ Server or client node that may be used as a target for a
service deployment
▸ Can deploy 1 or more services
▸ Separately updated and maintained
CLUSTER COMMUNICATION - 3 TYPES
▸ In-Cluster Communication w/Service Grid API
▸ Define interface & methods supported by your service
▸ Service Interface - *must be in class path of all your service
nodes*
▸ Service Nodes & Applications
▸ External Communication Protocols
▸ Doesn’t know about Apache Ignite
▸ REST, Sockets…
INTERNAL AND EXTERNAL APPS
▸ Internal Apps
▸ Connect via Ignite API
▸ Use Service Grid API
▸ External Apps
▸ Ignite “agnostic”
▸ Open endpoints - REST, sockets…whatever.
PERSISTENT STORAGE
▸ Data Nodes can connect to:
▸ Relational DBs
▸ NoSQL Das like Cassandra & MongoDB (ew)
▸ Hadoop
▸ Etc.
▸ Read-Through
▸ Write-Through
▸ Transactional
▸ Dependent on what type of persistent store you have elected
WRITE-THROUGH
BEST PLACE TO
GET STARTED?
https://console.gridgain.com/
REPEAT AFTER
ME…
CACHE
RULES
EVERYTHING
AROUND
ME
WHAT WE REALLY
MEANT IN THE SONG
THANKS!!!
DANI TRAPHAGEN - @DTRAPEZOID

More Related Content

What's hot

Continuous Machine and Deep Learning with Apache Ignite
Continuous Machine and Deep Learning with Apache IgniteContinuous Machine and Deep Learning with Apache Ignite
Continuous Machine and Deep Learning with Apache Ignite
Denis Magda
 
Microservices Architectures With Apache Ignite
Microservices Architectures With Apache IgniteMicroservices Architectures With Apache Ignite
Microservices Architectures With Apache Ignite
Denis Magda
 
Apache Ignite: In-Memory Hammer for Your Data Science Toolkit
Apache Ignite: In-Memory Hammer for Your Data Science ToolkitApache Ignite: In-Memory Hammer for Your Data Science Toolkit
Apache Ignite: In-Memory Hammer for Your Data Science Toolkit
Denis Magda
 
In-Memory Computing Essentials for Software Engineers
In-Memory Computing Essentials for Software EngineersIn-Memory Computing Essentials for Software Engineers
In-Memory Computing Essentials for Software Engineers
Denis Magda
 
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
DataWorks Summit/Hadoop Summit
 
In-Memory Computing Essentials
In-Memory Computing EssentialsIn-Memory Computing Essentials
In-Memory Computing Essentials
Denis Magda
 
In-Memory Computing Essentials for Architects and Engineers
In-Memory Computing Essentials for Architects and EngineersIn-Memory Computing Essentials for Architects and Engineers
In-Memory Computing Essentials for Architects and Engineers
Denis Magda
 
Presto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy endPresto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy end
Alluxio, Inc.
 
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
Codemotion
 
Challenges for running Hadoop on AWS - AdvancedAWS Meetup
Challenges for running Hadoop on AWS - AdvancedAWS MeetupChallenges for running Hadoop on AWS - AdvancedAWS Meetup
Challenges for running Hadoop on AWS - AdvancedAWS Meetup
Andrei Savu
 
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and BigtopAccelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
In-Memory Computing Summit
 
Google Cloud Platform
Google Cloud PlatformGoogle Cloud Platform
Google Cloud Platform
VMware Tanzu
 
Spark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattleSpark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattle
Judy Nash
 
10 Things About Spark
10 Things About Spark 10 Things About Spark
10 Things About Spark
Roger Brinkley
 
Building Modern Data Pipelines on GCP via a FREE online Bootcamp
Building Modern Data Pipelines on GCP via a FREE online BootcampBuilding Modern Data Pipelines on GCP via a FREE online Bootcamp
Building Modern Data Pipelines on GCP via a FREE online Bootcamp
Data Con LA
 
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Edge to AI:  Analytics from Edge to Cloud with Efficient Movement of Machine ...Edge to AI:  Analytics from Edge to Cloud with Efficient Movement of Machine ...
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Timothy Spann
 
Cloud computing by Google Cloud Platform - Presentation
Cloud computing by Google Cloud Platform - PresentationCloud computing by Google Cloud Platform - Presentation
Cloud computing by Google Cloud Platform - Presentation
TinarivosoaAbaniaina
 
Big data on google cloud
Big data on google cloudBig data on google cloud
Big data on google cloud
Tu Pham
 
Azure Databricks—Apache Spark as a Service with Sascha Dittmann
Azure Databricks—Apache Spark as a Service with Sascha DittmannAzure Databricks—Apache Spark as a Service with Sascha Dittmann
Azure Databricks—Apache Spark as a Service with Sascha Dittmann
Databricks
 
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
Timothy Spann
 

What's hot (20)

Continuous Machine and Deep Learning with Apache Ignite
Continuous Machine and Deep Learning with Apache IgniteContinuous Machine and Deep Learning with Apache Ignite
Continuous Machine and Deep Learning with Apache Ignite
 
Microservices Architectures With Apache Ignite
Microservices Architectures With Apache IgniteMicroservices Architectures With Apache Ignite
Microservices Architectures With Apache Ignite
 
Apache Ignite: In-Memory Hammer for Your Data Science Toolkit
Apache Ignite: In-Memory Hammer for Your Data Science ToolkitApache Ignite: In-Memory Hammer for Your Data Science Toolkit
Apache Ignite: In-Memory Hammer for Your Data Science Toolkit
 
In-Memory Computing Essentials for Software Engineers
In-Memory Computing Essentials for Software EngineersIn-Memory Computing Essentials for Software Engineers
In-Memory Computing Essentials for Software Engineers
 
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
Open Source Ingredients for Interactive Data Analysis in Spark by Maxim Lukiy...
 
In-Memory Computing Essentials
In-Memory Computing EssentialsIn-Memory Computing Essentials
In-Memory Computing Essentials
 
In-Memory Computing Essentials for Architects and Engineers
In-Memory Computing Essentials for Architects and EngineersIn-Memory Computing Essentials for Architects and Engineers
In-Memory Computing Essentials for Architects and Engineers
 
Presto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy endPresto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy end
 
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017
 
Challenges for running Hadoop on AWS - AdvancedAWS Meetup
Challenges for running Hadoop on AWS - AdvancedAWS MeetupChallenges for running Hadoop on AWS - AdvancedAWS Meetup
Challenges for running Hadoop on AWS - AdvancedAWS Meetup
 
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and BigtopAccelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
Accelerating the Hadoop data stack with Apache Ignite, Spark and Bigtop
 
Google Cloud Platform
Google Cloud PlatformGoogle Cloud Platform
Google Cloud Platform
 
Spark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattleSpark on Azure HDInsight - spark meetup seattle
Spark on Azure HDInsight - spark meetup seattle
 
10 Things About Spark
10 Things About Spark 10 Things About Spark
10 Things About Spark
 
Building Modern Data Pipelines on GCP via a FREE online Bootcamp
Building Modern Data Pipelines on GCP via a FREE online BootcampBuilding Modern Data Pipelines on GCP via a FREE online Bootcamp
Building Modern Data Pipelines on GCP via a FREE online Bootcamp
 
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Edge to AI:  Analytics from Edge to Cloud with Efficient Movement of Machine ...Edge to AI:  Analytics from Edge to Cloud with Efficient Movement of Machine ...
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
 
Cloud computing by Google Cloud Platform - Presentation
Cloud computing by Google Cloud Platform - PresentationCloud computing by Google Cloud Platform - Presentation
Cloud computing by Google Cloud Platform - Presentation
 
Big data on google cloud
Big data on google cloudBig data on google cloud
Big data on google cloud
 
Azure Databricks—Apache Spark as a Service with Sascha Dittmann
Azure Databricks—Apache Spark as a Service with Sascha DittmannAzure Databricks—Apache Spark as a Service with Sascha Dittmann
Azure Databricks—Apache Spark as a Service with Sascha Dittmann
 
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
Mm.. FLaNK Stack (MiNiFi MXNet Flink NiFi Kudu Kafka)
 

Similar to The next-phase-of-distributed-systems-with-apache-ignite

Nike tech-talk-intro-to-apache-ignite
Nike tech-talk-intro-to-apache-igniteNike tech-talk-intro-to-apache-ignite
Nike tech-talk-intro-to-apache-ignite
Dani Traphagen
 
Distributed Tracing
Distributed TracingDistributed Tracing
Distributed Tracing
Kevin Ingelman
 
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
Andre Essing
 
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
AboutYouGmbH
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
Sunil Govindan
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
Sunil Govindan
 
Building a scalable analytics environment to support diverse workloads
Building a scalable analytics environment to support diverse workloadsBuilding a scalable analytics environment to support diverse workloads
Building a scalable analytics environment to support diverse workloads
Alluxio, Inc.
 
GraphQL And Relay Modern
GraphQL And Relay ModernGraphQL And Relay Modern
GraphQL And Relay Modern
Brad Pillow
 
GraphQL and Relay Modern
GraphQL and Relay ModernGraphQL and Relay Modern
GraphQL and Relay Modern
Carmel JavaScript Roundabout
 
GraphQL And Relay Modern
GraphQL And Relay ModernGraphQL And Relay Modern
GraphQL And Relay Modern
Brad Pillow
 
How to Design a Backend for IoT
How to Design a Backend for IoTHow to Design a Backend for IoT
How to Design a Backend for IoT
İbrahim Gürses
 
INTERFACE, by apidays - C* made easy with Stargate APIs by Kirsten Hunter, D...
INTERFACE, by apidays  - C* made easy with Stargate APIs by Kirsten Hunter, D...INTERFACE, by apidays  - C* made easy with Stargate APIs by Kirsten Hunter, D...
INTERFACE, by apidays - C* made easy with Stargate APIs by Kirsten Hunter, D...
apidays
 
The Next Big Thing: Serverless
The Next Big Thing: ServerlessThe Next Big Thing: Serverless
The Next Big Thing: Serverless
Doug Vanderweide
 
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New York
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New YorkInfinitely Scalable Clusters - Grid Computing on Public Cloud - New York
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New York
Hentsū
 
Azure Cosmos DB - Technical Deep Dive
Azure Cosmos DB - Technical Deep DiveAzure Cosmos DB - Technical Deep Dive
Azure Cosmos DB - Technical Deep Dive
Andre Essing
 
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
ScyllaDB
 
Cassandra's Odyssey @ Netflix
Cassandra's Odyssey @ NetflixCassandra's Odyssey @ Netflix
Cassandra's Odyssey @ Netflix
Roopa Tangirala
 
AWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data AnalyticsAWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data Analytics
Amazon Web Services
 
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq AbdullahLeveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
Databricks
 
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ PanoraysQuick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Demi Ben-Ari
 

Similar to The next-phase-of-distributed-systems-with-apache-ignite (20)

Nike tech-talk-intro-to-apache-ignite
Nike tech-talk-intro-to-apache-igniteNike tech-talk-intro-to-apache-ignite
Nike tech-talk-intro-to-apache-ignite
 
Distributed Tracing
Distributed TracingDistributed Tracing
Distributed Tracing
 
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
Azure Cosmos DB - NoSQL Strikes Back (An introduction to the dark side of you...
 
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
Artur Borycki - Beyond Lambda - how to get from logical to physical - code.ta...
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Building a scalable analytics environment to support diverse workloads
Building a scalable analytics environment to support diverse workloadsBuilding a scalable analytics environment to support diverse workloads
Building a scalable analytics environment to support diverse workloads
 
GraphQL And Relay Modern
GraphQL And Relay ModernGraphQL And Relay Modern
GraphQL And Relay Modern
 
GraphQL and Relay Modern
GraphQL and Relay ModernGraphQL and Relay Modern
GraphQL and Relay Modern
 
GraphQL And Relay Modern
GraphQL And Relay ModernGraphQL And Relay Modern
GraphQL And Relay Modern
 
How to Design a Backend for IoT
How to Design a Backend for IoTHow to Design a Backend for IoT
How to Design a Backend for IoT
 
INTERFACE, by apidays - C* made easy with Stargate APIs by Kirsten Hunter, D...
INTERFACE, by apidays  - C* made easy with Stargate APIs by Kirsten Hunter, D...INTERFACE, by apidays  - C* made easy with Stargate APIs by Kirsten Hunter, D...
INTERFACE, by apidays - C* made easy with Stargate APIs by Kirsten Hunter, D...
 
The Next Big Thing: Serverless
The Next Big Thing: ServerlessThe Next Big Thing: Serverless
The Next Big Thing: Serverless
 
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New York
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New YorkInfinitely Scalable Clusters - Grid Computing on Public Cloud - New York
Infinitely Scalable Clusters - Grid Computing on Public Cloud - New York
 
Azure Cosmos DB - Technical Deep Dive
Azure Cosmos DB - Technical Deep DiveAzure Cosmos DB - Technical Deep Dive
Azure Cosmos DB - Technical Deep Dive
 
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
 
Cassandra's Odyssey @ Netflix
Cassandra's Odyssey @ NetflixCassandra's Odyssey @ Netflix
Cassandra's Odyssey @ Netflix
 
AWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data AnalyticsAWS Sydney Summit 2013 - Big Data Analytics
AWS Sydney Summit 2013 - Big Data Analytics
 
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq AbdullahLeveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq Abdullah
 
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ PanoraysQuick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
 

More from Dani Traphagen

Kafka + Kubernetes + why you maybe should
Kafka + Kubernetes + why you maybe shouldKafka + Kubernetes + why you maybe should
Kafka + Kubernetes + why you maybe should
Dani Traphagen
 
To Ksql Or Live the KStream
To Ksql Or Live the KStreamTo Ksql Or Live the KStream
To Ksql Or Live the KStream
Dani Traphagen
 
Sf k8-ignite-meetup
Sf k8-ignite-meetupSf k8-ignite-meetup
Sf k8-ignite-meetup
Dani Traphagen
 
Verizon k8-ignite-meetup
Verizon k8-ignite-meetupVerizon k8-ignite-meetup
Verizon k8-ignite-meetup
Dani Traphagen
 
Deploy like a Boss: Using Kubernetes and Apache Ignite!
Deploy like a Boss: Using Kubernetes and Apache Ignite!Deploy like a Boss: Using Kubernetes and Apache Ignite!
Deploy like a Boss: Using Kubernetes and Apache Ignite!
Dani Traphagen
 
Data Modeling with Cassandra and Time Series Data
Data Modeling with Cassandra and Time Series DataData Modeling with Cassandra and Time Series Data
Data Modeling with Cassandra and Time Series Data
Dani Traphagen
 
Diving into DSE Graph
Diving into DSE Graph Diving into DSE Graph
Diving into DSE Graph
Dani Traphagen
 
OSCON TALK: Becoming Friends with Cassandra and Spark
OSCON TALK: Becoming Friends with Cassandra and SparkOSCON TALK: Becoming Friends with Cassandra and Spark
OSCON TALK: Becoming Friends with Cassandra and Spark
Dani Traphagen
 

More from Dani Traphagen (8)

Kafka + Kubernetes + why you maybe should
Kafka + Kubernetes + why you maybe shouldKafka + Kubernetes + why you maybe should
Kafka + Kubernetes + why you maybe should
 
To Ksql Or Live the KStream
To Ksql Or Live the KStreamTo Ksql Or Live the KStream
To Ksql Or Live the KStream
 
Sf k8-ignite-meetup
Sf k8-ignite-meetupSf k8-ignite-meetup
Sf k8-ignite-meetup
 
Verizon k8-ignite-meetup
Verizon k8-ignite-meetupVerizon k8-ignite-meetup
Verizon k8-ignite-meetup
 
Deploy like a Boss: Using Kubernetes and Apache Ignite!
Deploy like a Boss: Using Kubernetes and Apache Ignite!Deploy like a Boss: Using Kubernetes and Apache Ignite!
Deploy like a Boss: Using Kubernetes and Apache Ignite!
 
Data Modeling with Cassandra and Time Series Data
Data Modeling with Cassandra and Time Series DataData Modeling with Cassandra and Time Series Data
Data Modeling with Cassandra and Time Series Data
 
Diving into DSE Graph
Diving into DSE Graph Diving into DSE Graph
Diving into DSE Graph
 
OSCON TALK: Becoming Friends with Cassandra and Spark
OSCON TALK: Becoming Friends with Cassandra and SparkOSCON TALK: Becoming Friends with Cassandra and Spark
OSCON TALK: Becoming Friends with Cassandra and Spark
 

Recently uploaded

Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 

Recently uploaded (20)

Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 

The next-phase-of-distributed-systems-with-apache-ignite

  • 1. APACHE IGNITE THE NEXT PHASE OF DISTRIBUTED SYSTEMS WITH
  • 2. OMG IT’S LIKE TOTALLY ME…. DANI TRAPHAGEN - @DTRAPEZOID ▸ The person in front of the room ▸ This doesn’t make me more special than you ▸ This just means I spent an afternoon of my life submitting an abstract ▸ No, but seriously, I did that ▸ But here’s a picture of me looking totes professional ▸ Oh also, I used to consult on NoSQL databases and now I don’t…because I sort of question………….
  • 6. SO THE TREND IS MEMORY IS DECREASING IN COST. COOL.
  • 7. SO WHAT IN THE H-E- IS APACHE IGNITE?
  • 9. A WEE BIT OF HISTORY APACHE IGNITE’S HISTORY ▸ Apache Ignite hits the streets in Oct. 2014, released by GridGain ▸ Aug. 2015 Ignite is the 2nd fastest project to graduate after Spark ▸ Today, over 100 Contributors internationally & rapidly growing out the community ▸ Over 1m lines of code
  • 12. ‣ HPC ‣ Machine learning ‣ Risk analysis ‣ Grid computing ‣ HA API Services ‣ Scalable Middleware ‣ Distributed caching ‣ KeyValue Store ‣ In-Memory SQL ‣ APIs for Kafka, etc.,as well as Ignite’s ‣ Lambda Architectures ‣ Big Data ‣ Realtime Analytics ‣ Batch processing ‣ Distributed In- Memory File System ‣ Node2Node & Topic-based Messaging ‣ Fault Tolerance ‣ Multiple backups ‣ Cluster groups ‣ Auto ‣ CEP,Streaming Windows & Cont.Queries ‣ Event driven design ‣ Dist.queues ‣ Atomic variables ‣ Dist. Semaphore APACHE IGNITE IS MORE THAN A BASIC CACHE, IT’S A PLATFORM
  • 13. DATA GRID This is the foundation of Ignite - rebalances the data in a load balanced and fault tolerant fashion
  • 14. KEY VALUE STORE & HIGH AVAILABILITY WITH APACHE IGNITE ▸ Ignite’s Data Grid is a Distributed Key Value Store, Caching Distributed In-Memory Data Throughout the Cluster ▸ Two Flavors of Cache: Partitioned or Replicated ▸ Consistency ▸ Data Locality
  • 16. MICROSERVICES DEPLOYMENT WITH APACHE IGNITE ▸ The Service Grid is the foundation for a Microservices- based Architecture ▸ Aspects of the Microservices Solution ▸ Data Nodes, Service Nodes ▸ Communication ▸ Internal and External Apps ▸ Persistence
  • 18. ▸ Deployment - Node Filter - a class that allows you to specify nodes to run your service ▸ Service Grid APIs - Pass Service Service Proxy ▸ Load Balancing - Sticky vs. Non-Sticky ▸ Manageable Deployment - Not necessary everywhere in your cluster, only where you need services deployment
  • 19. RECAP ▸ Manage the Lifecycle of your Services ▸ Initiate, Maintain, Kill ▸ Load Balancing ▸ Requests should be going to the services you have ▸ Fault Tolerance ▸ If a service goes down, it should be addressed immediately ▸ Scalability ▸ If volume demands change, you need to be able to handle it
  • 20. Persistent Storage EXTERNAL MICROSERVICES APPLICATIONS DATA NODE DATA NODE DATA NODE DATA NODE Service Node Service Node Service Node MS1MS1 MS2MS3MS2
  • 21. DATA NODES - ▸ Server nodes that store the data & accepts queries (& or computations) in a distributed fashion ▸ Provides distributed storage, don’t need provide data model classes, services classes.
  • 22. SERVICE NODES ▸ Server or client node that may be used as a target for a service deployment ▸ Can deploy 1 or more services ▸ Separately updated and maintained
  • 23. CLUSTER COMMUNICATION - 3 TYPES ▸ In-Cluster Communication w/Service Grid API ▸ Define interface & methods supported by your service ▸ Service Interface - *must be in class path of all your service nodes* ▸ Service Nodes & Applications ▸ External Communication Protocols ▸ Doesn’t know about Apache Ignite ▸ REST, Sockets…
  • 24. INTERNAL AND EXTERNAL APPS ▸ Internal Apps ▸ Connect via Ignite API ▸ Use Service Grid API ▸ External Apps ▸ Ignite “agnostic” ▸ Open endpoints - REST, sockets…whatever.
  • 25. PERSISTENT STORAGE ▸ Data Nodes can connect to: ▸ Relational DBs ▸ NoSQL Das like Cassandra & MongoDB (ew) ▸ Hadoop ▸ Etc. ▸ Read-Through ▸ Write-Through ▸ Transactional ▸ Dependent on what type of persistent store you have elected
  • 27. BEST PLACE TO GET STARTED? https://console.gridgain.com/