S U R I N D E R
2 9 T H J U L Y , 2 0 1 6
In-Memory Data Fabric
Apache Ignite 1.5
Data Fabric
Introduction
 A collection of independent, well-integrated, in-
memory components
 high-performance, integrated and distributed in-
memory platform
 transacting on large-scale data sets in real-time
 faster than possible with traditional disk-based
or flash technologies
Data Grid
 Distributed In-Memory
Caching
 Implements JCache (JSR
107)
 Distributed In-Memory
Transactions
 Data Consistency
 Tiered Off-Heap Storage
 Distributed ANSI-99
SQL Queries with
support for Joins
Keep required backup
Everyone knows
everything
Cache Modes
Cache Queries…
 Scan Query : return data matching BiPredicate
 Predicate sent to each node,
 Node scan its cache
 Data consolidated by requested node
 Sql Query : load data based on sql given
 Needs indexing to be enabled
 Registering indexing in config
 Annotations for fields visibility
 H2 Console for debugging
 Text Query : Query cache object on given value
 TextQuery txt = new TextQuery(Person.class, "Master Degree");
Cache Queries contd..
 Continuous Query : listens to change in data that fall
into filter
 Initial Query(any of fro previous slide)
 Remote Filter : evaluate the key on primary node to notify
local listener about the changes
 Local Listener : Perform the intended task when it get
notification about the change in data.
Off Heap Memory
 data can be stored and moved between on-
heap, off-heap, and swap space
 On Heap Tiered : stores on heap, evict to off heap
and then may evict to swap space(default mode)
 Off Heap Tiered : bypass on heap, may evict to swap
space
 Off Heap Values : Keys On heap and values off
Heap.
 Useful when keys are small and values are huge
 Swap Space : Disk space to evict data exceeding on
heap and off heap
Eviction Policies
 LRU : recommended when in doubt
 FIFO : it ignores the element access order
 Random
 Randomly evict any element
 Used for debugging
 Not beneficial when data fits in memory
Persistent Store
 CacheStore implements cacheLoader and
cacheWriter from JCache
 Read through
 Write through
 Write behind
 Works behind the cache API’s
Data Rebalancing
 Used when new node join the grid
 Possibly more backups than configured in such scenarios
 Rebalance Modes
 SYNC: cache calls blocked until rebalancing is completed
 ASYNC: rebalancing happen in background. Cache respond immediately
 NONE : No rebalancing, cache loaded on demand when required or
explicitly loading
Questions
Apache ignite Datagrid

Apache ignite Datagrid

  • 1.
    S U RI N D E R 2 9 T H J U L Y , 2 0 1 6 In-Memory Data Fabric Apache Ignite 1.5
  • 2.
  • 3.
    Introduction  A collectionof independent, well-integrated, in- memory components  high-performance, integrated and distributed in- memory platform  transacting on large-scale data sets in real-time  faster than possible with traditional disk-based or flash technologies
  • 4.
    Data Grid  DistributedIn-Memory Caching  Implements JCache (JSR 107)  Distributed In-Memory Transactions  Data Consistency  Tiered Off-Heap Storage  Distributed ANSI-99 SQL Queries with support for Joins
  • 5.
    Keep required backup Everyoneknows everything Cache Modes
  • 6.
    Cache Queries…  ScanQuery : return data matching BiPredicate  Predicate sent to each node,  Node scan its cache  Data consolidated by requested node  Sql Query : load data based on sql given  Needs indexing to be enabled  Registering indexing in config  Annotations for fields visibility  H2 Console for debugging  Text Query : Query cache object on given value  TextQuery txt = new TextQuery(Person.class, "Master Degree");
  • 7.
    Cache Queries contd.. Continuous Query : listens to change in data that fall into filter  Initial Query(any of fro previous slide)  Remote Filter : evaluate the key on primary node to notify local listener about the changes  Local Listener : Perform the intended task when it get notification about the change in data.
  • 8.
    Off Heap Memory data can be stored and moved between on- heap, off-heap, and swap space  On Heap Tiered : stores on heap, evict to off heap and then may evict to swap space(default mode)  Off Heap Tiered : bypass on heap, may evict to swap space  Off Heap Values : Keys On heap and values off Heap.  Useful when keys are small and values are huge  Swap Space : Disk space to evict data exceeding on heap and off heap
  • 9.
    Eviction Policies  LRU: recommended when in doubt  FIFO : it ignores the element access order  Random  Randomly evict any element  Used for debugging  Not beneficial when data fits in memory
  • 10.
    Persistent Store  CacheStoreimplements cacheLoader and cacheWriter from JCache  Read through  Write through  Write behind  Works behind the cache API’s
  • 11.
    Data Rebalancing  Usedwhen new node join the grid  Possibly more backups than configured in such scenarios  Rebalance Modes  SYNC: cache calls blocked until rebalancing is completed  ASYNC: rebalancing happen in background. Cache respond immediately  NONE : No rebalancing, cache loaded on demand when required or explicitly loading
  • 12.

Editor's Notes

  • #5 1. ability to add nodes on demand in real-time. 2. supports local, replicated, and partitioned data sets and allows to freely cross query between these data sets using standard SQL syntax 3. As long as your cluster is alive, Ignite will guarantee that the data between different cluster nodes will always remain consistent regardless of crashes or topology changes
  • #6 LocalMode : useful when data is not distributed to other caches. Ideal where data can be periodically refreshed at expiration frequency. By default, backups are turned off for better performance <property name="cacheMode" value="PARTITIONED"/