The Next Generation Application Server – How Event Based Processing yields scalability Guy Korland R&D Team Leader

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

About me… Core Team Leader – GigaSpaces since 2005 MSc – Technion (Prof. Roy Friedman) PhD candidate – Tel Aviv University (Prof. Nir Shavit) Lead Deuce STM – ( www.deucestm.org ) Java Software Transactional Memory

Core Team Leader – GigaSpaces since 2005

MSc – Technion (Prof. Roy Friedman)

PhD candidate – Tel Aviv University (Prof. Nir Shavit)

Lead Deuce STM – ( www.deucestm.org )

Java Software Transactional Memory

GigaSpaces XAP – Designed For: Performance Scalability Latency

GigaSpaces Evolution Single space Load Balance Partition & Replication SLA container Event Container NG Application Server PaaS Cloud 2000 2003 2005 2006 2007 2008 2009

Not going to talk about… Jini (Java SOA) Data Grid implementation. Map-Reduce. JDBC/JMS/JPA. Cloud computing. Batch processing. Mule ESB. WAN vs LAN Different languages interoperability. TupleSpace model extension. JDK improvements (RMI, Reflection, Serialization, Classloading…)

Jini (Java SOA)

Data Grid implementation.

Map-Reduce.

JDBC/JMS/JPA.

Cloud computing.

Batch processing.

Mule ESB.

WAN vs LAN

Different languages interoperability.

TupleSpace model extension.

JDK improvements (RMI, Reflection, Serialization, Classloading…)

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Today’s Reality – Tier Based Architecture Separate technology implementation Bottlenecks in all areas where state is stored, architecture can’t scale linearly! Separate technology implementation Separate technology implementation bottlenecks bottlenecks

Traditional Architecture - path to complexity… (marktplaats.nl) Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service Auction Service Bid Service Trade Service Info Service Timer Service T Bidder Validate Result Process Bid Bid Accepted Bid Result Process Trade Get Bid Result Place bid

Traditional Architecture - path to complexity… Business tier Back-up Back-up Redundancy doubles network traffic Bottlenecks are created Latency is increased Separate failover strategy and implementation for each tier Bidder Auction Owner Auction Service Bid Service Trade Service Info Service Timer Service

Redundancy doubles network traffic

Bottlenecks are created

Latency is increased

Separate failover strategy and implementation for each tier

Do you see the Problem? Business tier Scalability is not linear Scalability management nightmare Back-up Back-up Back-up Back-up Bidder Auction Owner

Scalability is not linear

Scalability management nightmare

There is a huge gap between peak and average loads

Bottlenecks, Performance, Scalability and High availability headaches Bad Publicity Revenue Loss Customer Dissatisfaction Regulatory Penalties

TBA – Summary • Historically the following has been done… – Tune, tune and tune configuration and code • Once a bottleneck has been resolved, the next one glooms – Hardware over provision • To make sure that at peak times the response times were still acceptable – Hardware upgrades • To get rid of bottlenecks, whose origin was impossible to track down – Alternative patterns • Avoiding 2-phase-commit, using patterns like ‘compensating transactions’ • Using Active/Passive failover, to make the response times faster, risking and in fact accepting potential data-loss • Partition the database, but not for size-reasons

• Historically the following has been done…

– Tune, tune and tune configuration and code

• Once a bottleneck has been resolved, the next one glooms

– Hardware over provision

• To make sure that at peak times the response times were still acceptable

– Hardware upgrades

• To get rid of bottlenecks, whose origin was impossible to track down

– Alternative patterns

• Avoiding 2-phase-commit, using patterns like ‘compensating transactions’

• Using Active/Passive failover, to make the response times faster, risking and

in fact accepting potential data-loss

• Partition the database, but not for size-reasons

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Event Containers

Based on JavaSpaces C++

The Master Worker Pattern

GigaSpaces - based on Shared Transactional Memory Read Write Write – writes a data object Read – reads a copy of a data object Take – reads a data object and deletes it Notify – generates an event on data updates Write Take Notify Write + Read  Data Caching Write + Notify  Messaging - Pub/Sub Write + Take  Master Worker

Write – writes a data object

Read – reads a copy of a data object

Take – reads a data object and deletes it

Notify – generates an event on data updates

Write + Read  Data Caching

Write + Notify  Messaging - Pub/Sub

Write + Take  Master Worker

Event Containers

Step 1 – Create a Processing Unit Business tier Processing Unit Single model for design, deployment and management No integration effort Manage data in memory Collapse the tiers Collocate the services Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner

Single model for design, deployment and management

No integration effort

Manage data in memory

Collapse the tiers

Collocate the services

Step 2 – Async Persistency Processing Unit Validate Process Bid Process Trade Process Results Persist for Compliance & Reporting purposes: - Storing State - Register Orders - etc. Collocation of data, messaging and services in memory: Minimum Latency (no network hops) Maximum Throughput Auction Service Bid Service Trade Service Info Service Timer Service Bidder Auction Owner Place Bid Get Bid Results

- Storing State - Register Orders - etc.

Collocation of data, messaging and services in memory:

Minimum Latency (no network hops)

Maximum Throughput

Step 3 – Resiliency Processing Unit Single, built-in failover/redundancy investment strategy Fewer points of failure Automated SLA driven failover/redundancy mechanism Continuous High Availability SLA Driven Container Backup

Single, built-in failover/redundancy investment strategy

Fewer points of failure

Automated SLA driven failover/redundancy mechanism

Continuous High Availability

Step 3 – Resiliency Processing Unit Automated SLA driven failover/redundancy mechanism Continuous Availability Self Healing Capability SLA Driven Container Backup Single, built-in failover/redundancy investment strategy Fewer integration points mean fewer chances for failure Primary Backup

Automated SLA driven failover/redundancy mechanism

Continuous Availability

Self Healing Capability

Single, built-in failover/redundancy investment strategy

Fewer integration points mean fewer chances for failure

Step 4 – Scale Processing Unit Write Once Scale Anywhere: Linear scalability Single monitoring and management engine Automated , SLA-Driven deployment and management Scaling policy, System requirements, Space cluster topology Backup Backup

Write Once Scale Anywhere:

Linear scalability

Single monitoring and management engine

Automated , SLA-Driven deployment and management

Scaling policy, System requirements, Space cluster topology

Event Containers

Step 5 – Auto Scale Out

Processing Unit – Scalability Unit Single Processing Unit Processing Unit - Scaled Involves Config Change No code changes!

Processing Unit – High-Availability Unit Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode

Database Integration - Async persistency Sync Replication Primary - Processing Unit Business logic – Active mode Backup - Processing Unit Business logic – Standby mode Mirror Process ORM Initial Load Async Replication Async Replication

XAP = Enterprise Grade Middleware Scale-out application server End 2 End scale-out middleware for: Web, Data, Messaging, Business logic Space Based Architecture – designed for scaling stateful applications In-memory Proven performance, Scalability, Low latency, Reliability SLA Driven Unique database scaling solution that fits cloud environment In Memory Data Grid O/R mapping support Support major Enterprise languages Java, .Net, C++

Scale-out application server

End 2 End scale-out middleware for: Web, Data, Messaging, Business logic

Space Based Architecture – designed for scaling stateful applications In-memory

Proven performance, Scalability, Low latency, Reliability

SLA Driven

Unique database scaling solution that fits cloud environment

In Memory Data Grid

O/R mapping support

Support major Enterprise languages

Java, .Net, C++

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Built-in Event Containers Polling Container Notify Container

Polling Container

Notify Container

Polling Container Used for point-to-point messaging Container polls the Space for events Comparable with the way Ajax works

Used for point-to-point messaging

Container polls the Space

for events

Comparable with the

way Ajax works

Notify Container Used for publish-subscribe messaging Space notifies the container

Used for publish-subscribe messaging

Space notifies the container

Typical Application

Service Grid Summary Powerful Universal Container Java/Net/C++ Distributed Fault Tolerant Object based Transactional Publish/Subscribe

Powerful Universal Container

Java/Net/C++

Distributed

Fault Tolerant

Object based

Transactional

Publish/Subscribe

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Event Containers

The POJO Based Data Domain Model @SpaceClass(fifo=true) public class Data { … @SpaceId ( autoGenerate = true ) public String getId () { return id; } public String setId ( String id ) { this .id = id; } public void setProcessed ( boolean processed ) { this . processed = processed; } public boolean isProcessed ( boolean processed ) { return this . processed; } } SpaceClass indicate that this is a SpaceEntry – SpaceClass includes classlevel attributes such as FIFO,Persistent… SpaceId used to define the key for that entry.

Data Processor Service Bean @SpaceDataEvent to be called when an event is triggered. public class DataProcessor{ @SpaceDataEvent public Data processData ( Data data ) { … data . setProcessed ( true ) ; //updates the space return data; } } Updates the data in the Space.

Wiring Order Processor Service Bean through Spring <bean id=&quot; dataProcessor “ class=&quot;com.gigaspaces.pu.example1.processor.DataProcessor&quot; /> <os-events:polling-container id=&quot;dataProcessorPollingEventContainer&quot; giga-space=&quot;gigaSpace&quot;> <os-events:tx-support tx-manager=&quot;transactionManager&quot;/> <os-core:template> <bean class=&quot;org.openspaces.example.data.common.Data&quot;> <property name=&quot;processed&quot; value=&quot;false&quot;/> </bean> </os-core:template> <os-events:listener> <os-events:annotation-adapter> <os-events:delegate ref=&quot; dataProcessor &quot;/> </os-events:annotation-adapter> </os-events:listener> </os-events:polling-container> The event Template The event Listener

Data Feeder public class DataFeeder { public void feed(){ Data data = new Data(counter++); data.setProcessed( false ); //feed data gigaSpace.write(data); } } Feed Data

Remoting – Taking one step forward Event

Remoting – Taking one step forward Reduce

Remoting – IDataProcessor Service API public interface IDataProcessor { // Process a given Data object Data processData(Data data); }

Remoting - DataProcessor Service @RemotingService public class DataProcessor implements IDataProcessor { public Data processData(Data data) { … data.setProcessed( true ); return data; } }

Remoting - Order Feeder public class DataFeeder { private IDataProcessor dataProcessor; public void setDataProcessor(…) { this .dataProcessor = dataProcessor; } public Data feed(){ Data data = new Data(counter++); // remoting call return dataProcessor.process (data) } }

Summary

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Scale up Throughput Benchmark – Physical Deployment Topology Remote (multiple machines , multiple processes) white box Client X4450 GigaSpaces 4 spaces , one per GSC X4450 GigaSpaces 4 spaces , one per GSC Switched Ethernet LAN Embedded (one machine , one process) X4450 Client GigaSpaces 8 spaces

Remote (multiple machines , multiple processes)

Scale up Throughput Benchmark – Embedded mode 1.8 Million read sec! 1.1 Million write/take sec!

Scale up Throughput Benchmark – Remote mode 90,00 read sec! 45,00 write/take sec!

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Event Containers

Web Container Grid

Web Container

Grid

Web application – Pet Clinic

Classic Architecture – Step 1- Request Submission Get request and invoke Service Data Grid

Classic Architecture – Step 2- Retrieve Results Page Generation Data Grid

Web Application Benchmark Results - Capacity

Web Application Benchmark Results - Capacity

Game Server

Game Server

Query Notify Intercepts update events Game Servers Table Feeder Loading Game Tables into the partitioned spaces Randomly updates the game tables Publisher (lobby) Game Table Directory Game Table search Player search Partitioned Space Pub/Sub messaging Scaling out GameTable Space Based Architecture – Game Server Publisher (II)

Loading Game Tables into the partitioned spaces

Randomly updates the game tables

Game Table Directory

Game Table search

Player search

Publisher Servers Game Servers GigaSpaces Service Grid Uploading 30,000 players for 6000 tables Randomly updates game tables Java runtime Physical backup Running continues query per user Space Based Architecture – Game Server Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable Notify / Query Notify / Query Partitioned Space GameTable Partitioned Space GameTable Partitioned Space GameTable

Uploading 30,000 players for 6000 tables

Randomly updates game tables

Running continues query per user

Dynamic repartitioning and load sharing I SLA Driven Container Indexed Notify / Query template Notify / Query template Partitioned Space Partitioned Space Partitioned Space

Dynamic repartitioning and load sharing II SLA Driven Container SLA Driven Container Partitioned Space Partitioned Space Partitioned Space

Scaling Throughput: ~6K/sec Throughput: ~12K/sec Throughput: ~18K/sec Partitioned Space Backup Space SLA Driven Container 6000 tables 30,000 players SLA Driven Container Partitioned Space Backup Space 4000 tables 20,000 players SLA Driven Container Partitioned Space Backup Space 2000 tables 10,000 players

6000 tables

30,000 players

4000 tables

20,000 players

2000 tables

10,000 players

Agenda Preview The Problem The Solution Event Containers Example Benchmarks Customer Use Cases Summary Q&A

Preview

The Problem

The Solution

Event Containers

Example

Benchmarks

Customer Use Cases

Summary Q&A

Thank You! Q&A

Appendix

SLA Driven Deployment SLA: Failover policy Scaling policy Ststem requirements Space cluster topology PU Services beans definition

SLA:

Failover policy

Scaling policy

Ststem requirements

Space cluster topology

PU Services beans definition

Continuous High Availability Fail-Over Failure

Dynamic Partitioning = Dynamic Capacity Growth VM 1 ,2G GSC VM 3 , 2G GSC VM 2 ,2G GSC Max Capacity=2G Max Capacity=4G Max Capacity=6G In some point VM 1 free memory is below 20 % - it about the time to increase the capacity – lets move Partitions 1 to another GSC and recover the data from the running backup! Later .. Partition 2 needs to move… After the move , data is recovered from the backup VM 5 , 4G GSC VM 4 ,4G GSC P - Primary B - Backup P P P B B B E F Partition 1 A B Partition 2 C D Partition 3 A B Partition 2 E F Partition 1 C D Partition 3

Executors

Executors

Task Executors – Task Execution Executing a task is done using the execute method AsyncFuture<Integer> future = gigaSpace .execute( new MyTask(2) ); int result = future.get(); 1 2 4 3

Executing a task is done using the execute method

Task Executors – Task Routing Routing a task can be done in three ways 1. Using the task itself 2. Passing a POJO to the execute method 3. Specifying a routing-parameter in the execute method

Routing a task can be done in three ways

1. Using the task itself

2. Passing a POJO to the execute method

3. Specifying a routing-parameter in the execute method

Task Executors – DistributedTask Execution Executing a distributed task is done using the execute method AsyncFuture<Integer> future = gigaSpace .execute( new MyDistTask() ); int result = future.get();

Executing a distributed task is done using the execute method

Task Executors – DistributedTask Routing Routing a distributed task can be done 1. In the same ways as with the plain Task interface 2. By broadcasting 3. Specifying a number of routing-parameters in the execute method

Routing a distributed task can be done

1. In the same ways as with the plain Task interface

2. By broadcasting

3. Specifying a number of routing-parameters in the execute method

Service Executors

Service Executors

IMDG Operations

IMDG

Operations

IMDG Basic Operations

IMDG Access – Space Operations – Write Operation write -operation writes a new object to a space Instantiate an object Set fields as necessary Write the object to the space Auction auction = new Auction(); auction.setType( &quot;Bicycle&quot; ); gigaSpace.write(auction);

write -operation writes a new object to a space

Instantiate an object

Set fields as necessary

Write the object to the space

IMDG Access – Space Operations – Read Operation read -operation reads an object from a space A copy of the object is returned The original copy remains in the space Build a template/query (more on this later) Read a matching object from the space Auction template = new Auction(); Auction returnedAuction = gigaSpace.read(template);

read -operation reads an object from a space

A copy of the object is returned

The original copy remains in the space

Build a template/query (more on this later)

Read a matching object from the space

Object SQL Query Support Supported Options and Queries Opeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN. GROUP BY – performs DISTINCT on the POJO properties Order By (ASC | DESC) SQLQuery rquery = new SQLQuery(MyPojo.class,&quot;firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'&quot;); Object[] result = space.readMultiple(rquery); Dynamic Query Support SQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?&quot;); query.setParameters(“david”,”lee”,50); Supported Options via JDBC API COUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases Join with 2 tables Non Supported HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON. NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK. Union, Minus, Union All. STDEV, STDEVP, VAR, VARP, FIRST, LAST. # LEFT , RIGHT [INNER] or [OUTER] JOIN

Supported Options and Queries

Opeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN.

GROUP BY – performs DISTINCT on the POJO properties

Order By (ASC | DESC)

SQLQuery rquery = new SQLQuery(MyPojo.class,&quot;firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'&quot;);

Object[] result = space.readMultiple(rquery);

Dynamic Query Support

SQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?&quot;);

query.setParameters(“david”,”lee”,50);

Supported Options via JDBC API

COUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases

Join with 2 tables

Non Supported

HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON.

NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK.

Union, Minus, Union All.

STDEV, STDEVP, VAR, VARP, FIRST, LAST.

# LEFT , RIGHT [INNER] or [OUTER] JOIN

IMDG Access – Space Operations – Take Operation take -operation takes an object from a space The matched object is removed from the space Build a template/query (more on this later) Take a matching object from the space Auction template = new Auction(); Auction removedAuction = gigaSpace.take(template);

take -operation takes an object from a space

The matched object is removed from the space

Build a template/query (more on this later)

Take a matching object from the space

IMDG Access – Space Operations – Update Operation update is equivalent to performing take and write Executed in a single atomic call AuctionItem item = new AuctionItem(); item.setType( &quot;Bicycle&quot; ); gigaSpace.write(item); item = gigaSpace.read(item); item.setType( &quot;Motorbike&quot; ); Object returnedObject = space.update(item, null , Lease.Forever, 2000L, UpdateModifiers.UPDATE_OR_WRITE);

update is equivalent to performing take and write

Executed in a single atomic call

IMDG Access – Space Operations – Batch API Apart from the single methods GigaSpaces also provides batch methods The methods are: writeMultiple : writes multiple objects readMultiple : reads multiple objects updateMultiple : updates multiple objects takeMultiple : reads multiple objects and deletes them Notes: Performance of the batch operations is generally higher Requires one call to the space Can be used with Template matching or SQLQuery

Apart from the single methods GigaSpaces also provides batch methods

The methods are:

writeMultiple : writes multiple objects

readMultiple : reads multiple objects

updateMultiple : updates multiple objects

takeMultiple : reads multiple objects and deletes them

Notes:

Performance of the batch operations is generally higher

Requires one call to the space

Can be used with Template matching or SQLQuery

IMDG Access – Space Operations – Batch API writeMultiple writes the specified objects to the space. Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.writeMultiple(auctions, 100);

writeMultiple writes the specified objects to the space.

IMDG Access – Space Operations – Batch API readMultiple reads all the objects matching the specified template from the space. Auction auction = new Auction(); Auction[] auctions = gigaSpace.readMultiple(auction, 100);

readMultiple reads all the objects matching the specified template from the space.

IMDG Access – Space Operations – Batch API takeMultiple takes all the objects matching the specified template from the space. Auction auction = new Auction(); Auction[] auctions = gigaSpace.takeMultiple(auction, 100);

takeMultiple takes all the objects matching the specified template from the space.

IMDG Access – Space Operations – Batch API updateMultiple updates a group of specified objects. Auction[] auctions = new Auction[] { new Auction(10), new Auction(20) }; auctions = gigaSpace.updateMultiple(auctions, 100);

updateMultiple updates a group of specified objects.

IMDG Summary Powerful shared memory Service Distributed Fault Tolerant Object based Single and Batch Operations Transactional

Powerful shared memory Service

Distributed

Fault Tolerant

Object based

Single and Batch Operations

Transactional