Getting Started with Apache Ignite as
a Distributed Database
Stephen Leung
Director of Solution Architect, Asia Pacific
2019 © GridGain Systems
2018 © GridGain Systems
Agenda
• Why not standard RDBMS and NoSQL?
• Apache Ignite Overview
• Reference Case sharing
• Q & A
2018 © GridGain Systems
Why not Standard RDBMS and NoSQL?
2018 © GridGain Systems
Existing Databases are Too Slow
2018 © GridGain Systems
Existing Databases Can't Scale
2018 © GridGain Systems
Existing Databases Can't Scale
2018 © GridGain Systems
NoSQL to the Rescue?
2018 © GridGain Systems
NoSQL to the Rescue?
RDBMS
Consistent,
but
not scalable
NoSQL
Scalable, but
not consistent
2018 © GridGain Systems
Apache Ignite
Consistent &
Scalable
2018 © GridGain Systems
Ignite
Data Warehouse Operational DB
TransactionsAnalytics, ML, AI
ETL
Analytics, ML, AI & Transactions
Real Time, Scalable, Available, Flexible
ETL, Batch, Inflexible
HTAP Architecture
2018 © GridGain Systems
In-Memory Computing
Mainframe NoSQL Hadoop
Data Layer
RDBMS
In-Memory Computing
Financial Services Initiatives
Private/Public
Cloud/SaaS
Regulatory
Compliance
Omnichannel
Banking
Security
Fraud
Risk
Management
Trading
HPC
2018 © GridGain Systems
Current solution comparison
Feature
In-Memory Cache
(Redis)
IMDG
(Hazelcast, GigaSpaces)
IMDB
(MemSQL, VoltDB)
IMC Platform
(GridGain)
Scale Out and Availability ✓ ✓ ✓ ✓
In-Memory ✓ ✓ ✓ ✓
Complements 3rd Party DB X ✓ X ✓
Persistence as Extension ✓ X
(copy of data in RAM)
X
(snapshots)
✓
Fast Restarts X X X ✓
Consistency and Transactions X ✓ ✓ ✓
SQL X X ✓ ✓
Collocated Processing X ✓ X ✓
Machine and Deep Learning ✓ X X ✓
2018 © GridGain Systems
Apache Ignite Overview
2018 © GridGain Systems
Memory-centric distributed
database, caching, and processing platform
In-memory computing platform based Apache Ignite
adding enterprise features and support
for mission critical deployments
2018 © GridGain Systems
15
Among Top Apache Projects
Top 5 Developer
Mailing Lists
1. Beam
2. Ignite
3. Kafka
4. Tomcat
5. James
Top 5 User
Mailing Lists
1. Flink
2. Lucene
3. Ignite
4. Cassandra
5. Kafka
Over 2M downloads per year
2018 © GridGain Systems
In-Memory Computing Benefits
In-Memory Speeds
10-1,000x faster than systems built on disk-based databases
Massive Scalability
Scale out to petabytes of in-memory data
Easy to Implement
No rip-and-replace of existing databases
2018 © GridGain Systems
Comprehensive Solution
• Slides In Between Existing
Application and Data Layers
• Works With RDBMS, NoSQL and
Hadoop Databases
• Multi-Language Support Including
SQL, Java, .NET, PHP, Node.js, Scala
and MapReduce
• Deploy On-Premises, In the Cloud, or
on Hybrid Environments
2018 © GridGain Systems
eCommerce, Retail & Travel
Financial Services
Software
FinTech Pharma & Healthcare
Ignite – Used by Leading Companies Worldwide
IoT AdTech
Telecom & Mobile
Logistics & Transportation
2018 © GridGain Systems
Apache Ignite In-Memory Computing Platform
Security&Auditing
Monitoring&Management
DataSnapshots&Recovery
Memory-Centric Storage
Scale to 1000s of Nodes & Store TBs of Data
Ignite Native Persistence
(Flash, SSD, Intel 3D XPoint)
Third-Party Persistence
Keep Your Own DB
(RDBMS, HDFS, NoSQL)
SQL Transactions Compute Services MLStreamingKey/Value
IoTFinancial
Services
Pharma &
Healthcare
E-CommerceTravel &
Logistics
Telco
DataCenterReplication
2018 © GridGain Systems
Distributed In-Memory Data Store
In-Memory Data Store
GridGain Server Cluster
Predictable Memory
Consumption
Fully Transactional WAL
(Write Ahead Log)
Instantaneous
Restarts
Automatic
Defragmentation
Off-heap Removes
Noticeable GC Pauses
Stores Superset
of Data
Distributed Persistent Store
In-Memory Data Store
Persistent Store
Server Node
In-Memory Data Store
Persistent Store
Server Node
In-Memory Data Store
Persistent Store
Server Node
2018 © GridGain Systems
Reference Case Sharing
2018 © GridGain Systems
The ING Group is a Dutch multinational banking
and financial services corporation headquartered
in Amsterdam. Its primary businesses are retail
banking, direct banking, commercial banking, investment
banking, asset management, and insurance services.
Problem
• To deliver new competitive customer services fast
• High cost of running on mainframe infrastructure
• Transaction consistency over multiple geo-locations
GridGain Solution
• Powers the core solution for delivering new services
Data aggregation across multiple sources
• Reduces infrastructure costs
Front-End APIs
Payments SecuritiesAccounts Credits Clients
Multi-datacenter Infrastructure
GridGain In-Memory Computing Platform
- Next Generation Banking
2018 © GridGain Systems
Wellington - Next Generation, Real-time IBOR
A top 20 worldwide asset management firm
with over $1 trillion under management
• Problem
– Current systems no longer scaled to handle the volumes
– Didn’t comply with new regulations following financial crisis
– Needed to introduce new asset classes faster
• GridGain Solution
– Investment Book of Record (IBOR), a single real-time
version of the truth for positions, exposure, valuations
and performance for all customers, teams and trades.
– 10x performance gains, linear horizontal scalability
– Support for SQL and ACID transactions, and for
existing systems and skillsets
– Enabled transactions and analytics on a single platform
– Collocated computing scales complex calculations, analytics
Trading
Systems
GridGain In-Memory Computing Platform
In-Memory
Data Grid
In-Memory
Database
Streaming
Analytics
Continuous
Learning Framework
Accounting
System
Other
Back Office
Portfolio
Management
Risk
Management
Regulatory &
Compliance
Investment
Book of Record
(IBOR)
Oracle RAC
2018 © GridGain Systems2019 © GridGain Systems
Ping An – Accelerate the database performance
24
Ping An Insurance has over 1M sales people
generating US$122.4B+ in revenue
Front-End APIs
SQL TXCompute
Service
Grid
Key /
Value
Commission Calculation System
GridGain In-Memory Computing Platform
Distributed In-Memory Data Store
Distributed Persistent Store
In-Memory Data Store
Persistent Store
Server Node
In-Memory Data Store
Persistent Store
Server Node
In-Memory Data Store
Persistent Store
Server Node
Problems
• Calculating sales commissions is a complex matrix math
challenge
• Key requirements: accuracy and speed
• Current platform: Oracle HW+SW for multi-field/range-
based queries across 8 or more tables
• Goal: achieve 5x improvement in calculation times—
without errors
GridGain Solution
• Reduce cost
– Move out from Oracle to GridGain
• Improve the delivery performance
– the commission calculation change from 8 hours to 1 hour.
• Immediate restart from disk (no memory warm-up)
2018 © GridGain Systems
What is Ignite?
- https://apacheignite.readme.io/docs/what-is-ignite
Getting Start
- https://apacheignite.readme.io/docs/getting-started
- In-memory computing essentials: Part 1
- In-memory computing essentials: Part 2
Download Apache Ignite
- https://ignite.apache.org/download.cgi
Free 30-Day Ultimate, Enterprise or Professional
Edition Trial
- https://www.gridgain.com/resources/download
Next Steps or Any Questions?

Getting Started with Apache Ignite as a Distributed Database

  • 1.
    Getting Started withApache Ignite as a Distributed Database Stephen Leung Director of Solution Architect, Asia Pacific 2019 © GridGain Systems
  • 2.
    2018 © GridGainSystems Agenda • Why not standard RDBMS and NoSQL? • Apache Ignite Overview • Reference Case sharing • Q & A
  • 3.
    2018 © GridGainSystems Why not Standard RDBMS and NoSQL?
  • 4.
    2018 © GridGainSystems Existing Databases are Too Slow
  • 5.
    2018 © GridGainSystems Existing Databases Can't Scale
  • 6.
    2018 © GridGainSystems Existing Databases Can't Scale
  • 7.
    2018 © GridGainSystems NoSQL to the Rescue?
  • 8.
    2018 © GridGainSystems NoSQL to the Rescue? RDBMS Consistent, but not scalable NoSQL Scalable, but not consistent
  • 9.
    2018 © GridGainSystems Apache Ignite Consistent & Scalable
  • 10.
    2018 © GridGainSystems Ignite Data Warehouse Operational DB TransactionsAnalytics, ML, AI ETL Analytics, ML, AI & Transactions Real Time, Scalable, Available, Flexible ETL, Batch, Inflexible HTAP Architecture
  • 11.
    2018 © GridGainSystems In-Memory Computing Mainframe NoSQL Hadoop Data Layer RDBMS In-Memory Computing Financial Services Initiatives Private/Public Cloud/SaaS Regulatory Compliance Omnichannel Banking Security Fraud Risk Management Trading HPC
  • 12.
    2018 © GridGainSystems Current solution comparison Feature In-Memory Cache (Redis) IMDG (Hazelcast, GigaSpaces) IMDB (MemSQL, VoltDB) IMC Platform (GridGain) Scale Out and Availability ✓ ✓ ✓ ✓ In-Memory ✓ ✓ ✓ ✓ Complements 3rd Party DB X ✓ X ✓ Persistence as Extension ✓ X (copy of data in RAM) X (snapshots) ✓ Fast Restarts X X X ✓ Consistency and Transactions X ✓ ✓ ✓ SQL X X ✓ ✓ Collocated Processing X ✓ X ✓ Machine and Deep Learning ✓ X X ✓
  • 13.
    2018 © GridGainSystems Apache Ignite Overview
  • 14.
    2018 © GridGainSystems Memory-centric distributed database, caching, and processing platform In-memory computing platform based Apache Ignite adding enterprise features and support for mission critical deployments
  • 15.
    2018 © GridGainSystems 15 Among Top Apache Projects Top 5 Developer Mailing Lists 1. Beam 2. Ignite 3. Kafka 4. Tomcat 5. James Top 5 User Mailing Lists 1. Flink 2. Lucene 3. Ignite 4. Cassandra 5. Kafka Over 2M downloads per year
  • 16.
    2018 © GridGainSystems In-Memory Computing Benefits In-Memory Speeds 10-1,000x faster than systems built on disk-based databases Massive Scalability Scale out to petabytes of in-memory data Easy to Implement No rip-and-replace of existing databases
  • 17.
    2018 © GridGainSystems Comprehensive Solution • Slides In Between Existing Application and Data Layers • Works With RDBMS, NoSQL and Hadoop Databases • Multi-Language Support Including SQL, Java, .NET, PHP, Node.js, Scala and MapReduce • Deploy On-Premises, In the Cloud, or on Hybrid Environments
  • 18.
    2018 © GridGainSystems eCommerce, Retail & Travel Financial Services Software FinTech Pharma & Healthcare Ignite – Used by Leading Companies Worldwide IoT AdTech Telecom & Mobile Logistics & Transportation
  • 19.
    2018 © GridGainSystems Apache Ignite In-Memory Computing Platform Security&Auditing Monitoring&Management DataSnapshots&Recovery Memory-Centric Storage Scale to 1000s of Nodes & Store TBs of Data Ignite Native Persistence (Flash, SSD, Intel 3D XPoint) Third-Party Persistence Keep Your Own DB (RDBMS, HDFS, NoSQL) SQL Transactions Compute Services MLStreamingKey/Value IoTFinancial Services Pharma & Healthcare E-CommerceTravel & Logistics Telco DataCenterReplication
  • 20.
    2018 © GridGainSystems Distributed In-Memory Data Store In-Memory Data Store GridGain Server Cluster Predictable Memory Consumption Fully Transactional WAL (Write Ahead Log) Instantaneous Restarts Automatic Defragmentation Off-heap Removes Noticeable GC Pauses Stores Superset of Data Distributed Persistent Store In-Memory Data Store Persistent Store Server Node In-Memory Data Store Persistent Store Server Node In-Memory Data Store Persistent Store Server Node
  • 21.
    2018 © GridGainSystems Reference Case Sharing
  • 22.
    2018 © GridGainSystems The ING Group is a Dutch multinational banking and financial services corporation headquartered in Amsterdam. Its primary businesses are retail banking, direct banking, commercial banking, investment banking, asset management, and insurance services. Problem • To deliver new competitive customer services fast • High cost of running on mainframe infrastructure • Transaction consistency over multiple geo-locations GridGain Solution • Powers the core solution for delivering new services Data aggregation across multiple sources • Reduces infrastructure costs Front-End APIs Payments SecuritiesAccounts Credits Clients Multi-datacenter Infrastructure GridGain In-Memory Computing Platform - Next Generation Banking
  • 23.
    2018 © GridGainSystems Wellington - Next Generation, Real-time IBOR A top 20 worldwide asset management firm with over $1 trillion under management • Problem – Current systems no longer scaled to handle the volumes – Didn’t comply with new regulations following financial crisis – Needed to introduce new asset classes faster • GridGain Solution – Investment Book of Record (IBOR), a single real-time version of the truth for positions, exposure, valuations and performance for all customers, teams and trades. – 10x performance gains, linear horizontal scalability – Support for SQL and ACID transactions, and for existing systems and skillsets – Enabled transactions and analytics on a single platform – Collocated computing scales complex calculations, analytics Trading Systems GridGain In-Memory Computing Platform In-Memory Data Grid In-Memory Database Streaming Analytics Continuous Learning Framework Accounting System Other Back Office Portfolio Management Risk Management Regulatory & Compliance Investment Book of Record (IBOR) Oracle RAC
  • 24.
    2018 © GridGainSystems2019 © GridGain Systems Ping An – Accelerate the database performance 24 Ping An Insurance has over 1M sales people generating US$122.4B+ in revenue Front-End APIs SQL TXCompute Service Grid Key / Value Commission Calculation System GridGain In-Memory Computing Platform Distributed In-Memory Data Store Distributed Persistent Store In-Memory Data Store Persistent Store Server Node In-Memory Data Store Persistent Store Server Node In-Memory Data Store Persistent Store Server Node Problems • Calculating sales commissions is a complex matrix math challenge • Key requirements: accuracy and speed • Current platform: Oracle HW+SW for multi-field/range- based queries across 8 or more tables • Goal: achieve 5x improvement in calculation times— without errors GridGain Solution • Reduce cost – Move out from Oracle to GridGain • Improve the delivery performance – the commission calculation change from 8 hours to 1 hour. • Immediate restart from disk (no memory warm-up)
  • 25.
    2018 © GridGainSystems What is Ignite? - https://apacheignite.readme.io/docs/what-is-ignite Getting Start - https://apacheignite.readme.io/docs/getting-started - In-memory computing essentials: Part 1 - In-memory computing essentials: Part 2 Download Apache Ignite - https://ignite.apache.org/download.cgi Free 30-Day Ultimate, Enterprise or Professional Edition Trial - https://www.gridgain.com/resources/download Next Steps or Any Questions?