GigaSpaces Data Caching / Data Grid overview August 2009
<ul><li>Scaling Up Your Database by Adding a Data Grid </li></ul>
Scaling Up Your Database by Adding a Data Grid <ul><li>To scale up your database, use the IMDG directly </li></ul><ul><li>...
Benefits of using GigaSpaces as the system of record <ul><li>Decreasing database load  through partitioning and data distr...
IMDG Access support  <ul><li>Main Features: </li></ul><ul><ul><li>Direct persistency (Write/Read Through) </li></ul></ul><...
Step 2: Access data via IMDG SQL Queries <ul><li>Supported Options and Queries </li></ul><ul><ul><li>Opeations: =, <>, <,>...
<ul><li>GigaSpaces In-Memory-Data-Grid </li></ul>
The IMDG – Runtime Modes – Embedded <ul><li>An IMDG (space) instance that runs within the application memory address space...
The IMDG – Runtime Modes – Remote <ul><li>Accessing a remote space involves network calls and serialization/de-serializati...
The IMDG – Runtime Modes – Master-Local Cache <ul><li>A local ‘cache’  </li></ul><ul><ul><li>Embedded with a client </li><...
The IMDG – Runtime Modes – Master-Local View <ul><li>A local 'View' </li></ul><ul><ul><li>Embedded with a client </li></ul...
The IMDG – Runtime Modes – Persistent <ul><li>Stores data both into memory and on disk in a relational database </li></ul>...
Asynchronous Reliable Persistency - Write Behind <ul><li>The most common architecture </li></ul><ul><li>Database is out of...
The Initial Load – Fast Data load from the Database
<ul><li>IMDG </li></ul><ul><li>Deployment Topologies </li></ul>
IMDG Basic Deployment Topologies <ul><li>Primary-Backup </li></ul>Partitioned Feeder Feeder Partitioned + Backup Feeder
<ul><li>IMDG </li></ul><ul><li>Operations </li></ul>
IMDG Basic Operations
<ul><li>Move into SBA </li></ul>
What is Space Based Architecture (SBA) Space-Based Architecture  ( SBA ) is a  software architecture pattern  for achievin...
Move into SBA <ul><li>Deploy Application components as Processing Units </li></ul><ul><ul><li>Form a composite SOA Applica...
Space Based Architecture – Business logic and data collocated Pushing data into the backend system In-Memory-Data-Grid and...
Map-Reduce Approach to perform Parallel Query <ul><li>How The GigaSpaces Task Executors works? </li></ul><ul><ul><li>Phase...
Map-Reduce Approach to perform Parallel Query <ul><li>How Task Executors works?   </li></ul><ul><ul><li>Phase 2 - Getting ...
Distributed Task  Example <ul><li>public class MyDistTask implements DistributedTask<Integer, Long> { </li></ul><ul><li>pu...
<ul><li>SBA Fundamental </li></ul>
Space Based Architecture Fundamentals <ul><li>The CAP properties: </li></ul><ul><li>Strong  C onsistency:  </li></ul><ul><...
Using SBA to Virtualize the Middleware = GigaSpaces XAP <ul><li>Steps to virtualize the middleware: </li></ul><ul><ul><li>...
The Service Grid <ul><li>Continuous Application Availability To Achieve 99999’s  </li></ul><ul><ul><li>Automatically provi...
Typical Web Application Architecture Dynamic LB Configuration  Managed Jetty Web Containers,  Http Session on top of the S...
Upcoming SlideShare
Loading in …5
×

Giga Spaces Data Grid / Data Caching Overview

3,800
-1

Published on

An overview of the DataGrid and Data Caching capabilities of GigaSpaces XAP including topologies and patterns of use,

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • nice presentation, when the database becomes a performance bottleneck, the use of an in-memory distributed cache can be very good option. it is the easiest way to achieve high performance, scalability and availability. products like Giga Spaces XAP and NCache of Alachisoft (http://www.alachisoft.com/ncache/index.html) are doing very good job in this regard.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,800
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
100
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • XAP Features Scale-out application server End 2 End scale-out middleware for:   Web, Data, Messaging, Business logic Space Based Architecture – designed for scaling stateful applications In Memory Data Grid O/R mapping support Support major Enterprise languages Java, .Net C++
  • Giga Spaces Data Grid / Data Caching Overview

    1. 1. GigaSpaces Data Caching / Data Grid overview August 2009
    2. 2. <ul><li>Scaling Up Your Database by Adding a Data Grid </li></ul>
    3. 3. Scaling Up Your Database by Adding a Data Grid <ul><li>To scale up your database, use the IMDG directly </li></ul><ul><li>On the backend, the IMDG persists the data to your database using your existing Hibernate O/R mapping. </li></ul><ul><li>Hibernate used by the IMDG </li></ul><ul><li>Application using Native IMDG API - object/SQL API very similar to Hibernate </li></ul><ul><li>Gain full power of the IMDG </li></ul><ul><li>Good for write and read scenarios </li></ul>
    4. 4. Benefits of using GigaSpaces as the system of record <ul><li>Decreasing database load through partitioning and data distribution - enables higher data volumes and higher throughput with low latency </li></ul><ul><li>Better decoupling between your application and the database - no need to hard-wire Hibernate and database concepts into your code and runtime environment </li></ul><ul><li>Event-driven model enables notifications when data is modified </li></ul><ul><li>Database access can be synchronous or asynchronous - the GigaSpaces Mirror Service allows data to be persisted to the database asynchronously, without a performance penalty </li></ul>
    5. 5. IMDG Access support <ul><li>Main Features: </li></ul><ul><ul><li>Direct persistency (Write/Read Through) </li></ul></ul><ul><ul><li>Asynchronous Reliable persistency (Write Behind) </li></ul></ul><ul><ul><li>Fast Data load once IMDG started </li></ul></ul><ul><ul><li>Lazy load in case of a cache miss </li></ul></ul><ul><ul><li>Delegating IMDG SQL Queries to database </li></ul></ul><ul><ul><li>Advanced Hibernate and nHibernate integration </li></ul></ul><ul><ul><li>Java , C++ and .Net objects persistency </li></ul></ul><ul><ul><li>Custom persistency support </li></ul></ul>
    6. 6. Step 2: Access data via IMDG SQL Queries <ul><li>Supported Options and Queries </li></ul><ul><ul><li>Opeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN. </li></ul></ul><ul><ul><li>GROUP BY – performs DISTINCT on the POJO properties </li></ul></ul><ul><ul><li>Order By (ASC | DESC) </li></ul></ul><ul><li>SQLQuery rquery = new SQLQuery(MyPojo.class,&quot;firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'&quot;); </li></ul><ul><li>Object[] result = space.readMultiple(rquery); </li></ul><ul><li>Dynamic Query Support </li></ul><ul><ul><li>SQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?&quot;); </li></ul></ul><ul><ul><li>query.setParameters(“david”,”lee”,50); </li></ul></ul><ul><li>Supported Options via JDBC API </li></ul><ul><ul><li>COUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases </li></ul></ul><ul><ul><li>Join with 2 tables </li></ul></ul><ul><li>Non Supported </li></ul><ul><ul><li>HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON. </li></ul></ul><ul><ul><li>NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK. </li></ul></ul><ul><ul><li>Union, Minus, Union All. </li></ul></ul><ul><ul><li>STDEV, STDEVP, VAR, VARP, FIRST, LAST. </li></ul></ul><ul><ul><li># LEFT , RIGHT [INNER] or [OUTER] JOIN </li></ul></ul>
    7. 7. <ul><li>GigaSpaces In-Memory-Data-Grid </li></ul>
    8. 8. The IMDG – Runtime Modes – Embedded <ul><li>An IMDG (space) instance that runs within the application memory address space </li></ul><ul><li>Accessed by reference without going through network or serialization calls </li></ul><ul><li>Most efficient configuration mode </li></ul><ul><li>Used as the primary space configuration setup </li></ul>C++
    9. 9. The IMDG – Runtime Modes – Remote <ul><li>Accessing a remote space involves network calls and serialization/de-serialization of the cached objects between the client and the space process </li></ul><ul><li>Used only in cases where: </li></ul><ul><ul><li>Client application cannot run an embedded space (due to memory capacity limitations, etc.) </li></ul></ul><ul><ul><li>In cases where there are a large number of concurrent updates on the same cached object using different remote processes </li></ul></ul>C++ C++
    10. 10. The IMDG – Runtime Modes – Master-Local Cache <ul><li>A local ‘cache’ </li></ul><ul><ul><li>Embedded with a client </li></ul></ul><ul><ul><li>Set of cached objects is a snapshot </li></ul></ul><ul><ul><li>No additional objects get added to the local space unless new queries are made </li></ul></ul><ul><ul><li>Writes should be made on the master only </li></ul></ul><ul><li>Use when </li></ul><ul><ul><li>Many distributed clients </li></ul></ul><ul><ul><li>Accessing the same space </li></ul></ul><ul><ul><li>Read-mostly </li></ul></ul>
    11. 11. The IMDG – Runtime Modes – Master-Local View <ul><li>A local 'View' </li></ul><ul><ul><li>Embedded with a client </li></ul></ul><ul><ul><li>Contains updated and changing results based on a client specified query </li></ul></ul><ul><li>Used when </li></ul><ul><ul><li>Clients want to get a streaming view of a subset of the 'main' space </li></ul></ul><ul><li>Writes can be made to the view </li></ul><ul><ul><li>Contains a proxy to the master </li></ul></ul>
    12. 12. The IMDG – Runtime Modes – Persistent <ul><li>Stores data both into memory and on disk in a relational database </li></ul>Can use custom Mapping or built in Hibernate/nHibernate plug-in
    13. 13. Asynchronous Reliable Persistency - Write Behind <ul><li>The most common architecture </li></ul><ul><li>Database is out of the critical path of the transaction </li></ul><ul><li>IMDG operations and data are delegated to the database in a reliable, consistent manner </li></ul><ul><li>Support read and write scenarios </li></ul>Hibernate Hibernate Feeder
    14. 14. The Initial Load – Fast Data load from the Database
    15. 15. <ul><li>IMDG </li></ul><ul><li>Deployment Topologies </li></ul>
    16. 16. IMDG Basic Deployment Topologies <ul><li>Primary-Backup </li></ul>Partitioned Feeder Feeder Partitioned + Backup Feeder
    17. 17. <ul><li>IMDG </li></ul><ul><li>Operations </li></ul>
    18. 18. IMDG Basic Operations
    19. 19. <ul><li>Move into SBA </li></ul>
    20. 20. What is Space Based Architecture (SBA) Space-Based Architecture ( SBA ) is a software architecture pattern for achieving linear scalability of stateful, high-performance applications, based on Yale’s Tuple-Space Model (Source Wikipedia) <ul><li>What is a Processing Unit : </li></ul><ul><li>Bundle of services, data, messaging </li></ul><ul><li>Collocation into single VM </li></ul><ul><li>Unified Messaging & Data </li></ul><ul><li>In-Memory </li></ul><ul><li>Cloud of Processing Units </li></ul><ul><li>Scale through Partitioning </li></ul><ul><li>Virtualized middleware </li></ul><ul><li>What is a Space : </li></ul><ul><li>Elegant – 4 API </li></ul><ul><li>Solves: </li></ul><ul><ul><li>Data sharing </li></ul></ul><ul><ul><li>Messaging </li></ul></ul><ul><ul><li>Workflow </li></ul></ul><ul><ul><li>Parallel processing </li></ul></ul>
    21. 21. Move into SBA <ul><li>Deploy Application components as Processing Units </li></ul><ul><ul><li>Form a composite SOA Application </li></ul></ul><ul><li>Distributed Data Processing </li></ul><ul><ul><li>Use GigaSpaces event driven and data processing components to process incoming data in real time </li></ul></ul><ul><li>Collocate business logic and Data </li></ul><ul><ul><li>Scale these as one entity to allow true linear scalability </li></ul></ul>
    22. 22. Space Based Architecture – Business logic and data collocated Pushing data into the backend system In-Memory-Data-Grid and collocated Processing units Collects results / reporting Service Primary 1 Primary 2 Primary 3 Backup 3 Backup 2 Backup 1 Replication Replication Replication
    23. 23. Map-Reduce Approach to perform Parallel Query <ul><li>How The GigaSpaces Task Executors works? </li></ul><ul><ul><li>Phase 1 - Sending the Task to be executed: </li></ul></ul>
    24. 24. Map-Reduce Approach to perform Parallel Query <ul><li>How Task Executors works? </li></ul><ul><ul><li>Phase 2 - Getting the results back to be reduced. </li></ul></ul><ul><ul><li>The Task itself will query the IMDG instance and perform whatever calculations needed. </li></ul></ul>
    25. 25. Distributed Task Example <ul><li>public class MyDistTask implements DistributedTask<Integer, Long> { </li></ul><ul><li>public Integer execute() throws Exception { return 1; } </li></ul><ul><li>public Long reduce(List<AsyncResult<Integer>> results) throws Exception { </li></ul><ul><li>long sum = 0; </li></ul><ul><li>for (AsyncResult<Integer> result : results) { </li></ul><ul><li>if (result.getException() != null) { </li></ul><ul><li>throw result.getException(); </li></ul><ul><li>} </li></ul><ul><li>sum += result.getResult(); </li></ul><ul><li>} </li></ul><ul><li>return sum; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>AsyncFuture<Long> future = gigaSpace.execute(new MyDistTask()); long result = future.get(); // result will be the number of primary spaces The Task Reducer Implementation– Run at the client Side The Task execution – Called from the Client side The Task execute Implementation – Run at the Space Side
    26. 26. <ul><li>SBA Fundamental </li></ul>
    27. 27. Space Based Architecture Fundamentals <ul><li>The CAP properties: </li></ul><ul><li>Strong C onsistency: </li></ul><ul><ul><li>all clients see the same view, even in the presence of updates </li></ul></ul><ul><li>High A vailability: </li></ul><ul><ul><li>all clients can find some replica of the data, even in the presence of failures </li></ul></ul><ul><li>P artition-tolerance: </li></ul><ul><ul><li>the system properties hold even when the system is partitioned </li></ul></ul>Partitioned Data Grid Partitioned Data Grid with Backup Feeder Integrating with existing database
    28. 28. Using SBA to Virtualize the Middleware = GigaSpaces XAP <ul><li>Steps to virtualize the middleware: </li></ul><ul><ul><li>Decouple the application from the deployment environment </li></ul></ul><ul><ul><li>Use partitioning to split the load and the data </li></ul></ul><ul><ul><li>Move manual process to SLA driven deployment </li></ul></ul><ul><ul><li>Inject dynamic scaling and self healing </li></ul></ul><ul><li>The result: a scale-out application server providing: </li></ul><ul><ul><li>End-end scale-out middleware for Web data, messaging and business logic </li></ul></ul><ul><ul><li>In memory clustering </li></ul></ul><ul><ul><li>Unique database scalability </li></ul></ul><ul><ul><li>Automatic self healing </li></ul></ul><ul><li>Enterprise-grade and OEM-ready: </li></ul><ul><ul><li>Supports open-source and standard development frameworks </li></ul></ul><ul><ul><li>Supports Java, .NET, C++ and scripting languages </li></ul></ul>
    29. 29. The Service Grid <ul><li>Continuous Application Availability To Achieve 99999’s </li></ul><ul><ul><li>Automatically provision additional resources after failures </li></ul></ul><ul><li>Maintain optimal application performance </li></ul><ul><ul><li>Dynamically scale (or shrink) system resources based upon business demand </li></ul></ul><ul><li>Dramatic reduction in enterprise server utilization rates </li></ul><ul><ul><li>Dynamic provisioning eliminates the need to design for peak loads </li></ul></ul><ul><li>Significant reduction in IT Operations and system management costs </li></ul>An automated, SLA-based application provisioning & management engine
    30. 30. Typical Web Application Architecture Dynamic LB Configuration Managed Jetty Web Containers, Http Session on top of the Space Business Logic and Data on top of the Data Grid Interact with BL and Data via Space API, events, remoting or task executors Partitioning and collocation for best performance and scalability Async. Persistency Proactive Administration
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×