Sunx4450 Intel7460 GigaSpaces XAP Platform Benchmark

Ultra-Scalable and Blazing-Fast: The Sun Fire x4450-Intel 7460-XAP GigaSpaces Platform Scaling up with Commodity HW® Scale up Benchmark Report Shay Hassidim Deputy CTO GigaSpaces January 2009

Agenda
Benchmark targets
Results highlights
Servers used
Stack under test
The Sun Fire x4450
Intel® Xeon® Processor 7400 Series
SBA vs .TBA – Scaling vs. Latency and Costs
Scale up Benchmark Scenarios
Quick Introduction to GigaSpaces
Scale up Throughput Benchmark
Scale up Latency Benchmark
Web Application Benchmark
Risk Calculation Benchmark
Special tuning
Comparison of Six Core with Quad Core
Conclusions

Scaling up with commodity HW ® - Benchmark Target
“ S c a l i n g u p with commodity HW®”
Test how GigaSpaces based applications Scales-up on the Sun Fire x4450-Intel 7460-XAP GigaSpaces Platform and how the new Intel 45-nm technology with 4 CPUs and 6 cores utilized with mission critical applications

Scale up Benchmark Results Highlights - Throughput
Collocated space capacity
1.8 million read operations per/sec
1.1 million write/take operations per/sec
Less than 20% performance drop up to 16 threads
Remote space capacity
90,000 read operations per/sec
45,000 write/take operations per/sec (including HA).
Less than 20% performance drop up to 24 clients

Scale up Benchmark Results Highlights - Latency
1 ms latency with remote write operation
Load of 20,000 remote write operation per sec
including HA (with replication to backup)
Concurrent 20 clients
4 K payload.
0.4 ms latency with remote write operation
Load of 20,000 remote write operation per sec
excluding HA (without replication to backup)
Concurrent 20 clients
4 K payload.

Scale up Benchmark Results Highlights
Web Application
16,223 page generation/sec with 6 ms latency over LAN
3 web servers
2 x4450 machines
HPC Risk Calculation
Monte Carlo simulation
Near linear scalability (Up to 32 concurrent workers)
Boosting the calculation time in 2640% (compared to one worker)
Calculating 4096 portfolios in 100 seconds (41 calculation per/sec).

Servers Used Since we are running Scale up Benchmark we have fixed amount of machines - mySQL Database - Apache Load-Balancer 3.16GHz 4 2 (2 cores each) Intel Xeon 16 G RAM Sun Fire X4150 2 socket Sun 4 GigaSpaces Clients 2.66GHz 24 4 (6 cores each) Intel Dunnington X7460 32 G RAM 4 socket Intel White box 3 GigaSpaces 2.66GHz 24 4 (6 cores each) Intel Dunnington X7460 32 G RAM Sun Fire X4450 4 socket Sun 2 GigaSpaces 2.66GHz 24 4 (6 cores each) Intel Dunnington X7460 32 G RAM Sun Fire X4450 4 socket Sun 1 Running Clock speed # of Cores # of CPU CPU Type and Memory Model Vendor Server ID

Technology Stack under Test Ethernet (1gE) Sun Fire x4450 Intel Dunnington X7460 4 CPUs (6 cores each) Sun Solaris update 6 GigaSpaces XAP 6.2.2 Sun mySQL 5 Apache LB 2.2.9 Sun JDK 1.6

Sun Fire x4450
Comes with the Intel® Xeon® processor 7400 series with 6 processing cores.
Half the size of the other servers in its class
Industry's smallest enterprise class 4-socket x64 server
Supports up to 4 processors (24 cores), 8 SAS Disk Drives and 32 memory DIMMs in a 2RU form factor
Energy efficient design saves on power and cooling
Comprehensive and easy-to-use system management and monitoring built into every system
Choice of virtually any OS: Solaris, Linux, Windows, VMware
Key Applications
Server consolidation and virtualization
Database and web serving
Data warehousing and data analysis
Business processing

Intel® Xeon® Processor 7400 Series Benefit Feature
Flexibility for 64-bit and 32-bit applications and operating Systems
Intel® 64 architecture
Enables increased throughput and bandwidth between each of the processors and the chipset
1066 MHz Dedicated High-Speed Interconnects (DHSI)
A suite of processor hardware enhancements that assists virtualization software to deliver more efficient virtualization solutions and greater capabilities including 64-bit guest OS support
Intel® VT FlexPriority optimizes virtualization software efficiency by improving interrupt handling
Intel® VT FlexMigration enables Intel Xeon processor 7400 series-based systems to be added to the existing virtualization pool with single, two, or 4+ socket Intel Core microarchitecture-based servers
Intel® Virtualization Technology 3
Boosts performance on multiple applications/user environments and data-demanding workloads, while enabling denser data center deployments through improved performance-per-watt
45nm Hi-k process technology enables larger on-die cache for better performance, and reduced transistor
Increases efficiency of cache-to-core data transfers, maximizing main memory to processor bandwidth
Reduces latency by storing larger data sets closer to the processor, reducing the number of trips to main memory
Enhanced Intel® Core™ Microarchitecture 16 MB of L3 Cache
Platform-compatible with other Intel Xeon processor 7000-sequence processors for ease of migration and IT stability
Increased performance with 45nm technology and increased headroom for multi-threaded applications and data demanding applications
Enables improved virtualization performance, increasing system utilization
Multi-core processing

Scale up Benchmark Goals – Measuring Performance Scalability Latency

Scale up Benchmark Goals - Measure the Platform Performance , Scalability and Latency for
On-line Systems – Relevant for:
On-line Banking
On-line Trading
On-line Gaming
e-gov portals
e-commerce Retail
Logistics
Order-Management systems
Supply chain management
RFID Based Systems
HPC – Relevant for:
Risk Calculation
Real-time Analytics
Market Data
Large-volume data mining
Real-time tracking
Command-and-Control

SBA vs .TBA – Scaling vs. Latency and Costs SBA – Space Based Architecture TBA – Traditional Based Architecture

Scale up Benchmark Scenarios
Generic GigaSpaces XAP Benchmark:
Performance vs Scalability - Throughput and capacity. In proc and out proc benchmark. Additional client threads will be added to measure the scalability of the system.
Latency vs Scalability – Measuring the time it takes pushing data into GigaSpaces' in-memory-data-grid for processing under massive load.
Mission Critical applications Benchmark - On-Line web based and HPC applications benchmark:
Pet Clinic application - This application will be based on GigaSpaces as the backbone instead of a relational database.
Credit Risk Calculation – Monte Carlo simulation

Quick Introduction to GigaSpaces

Introduction - Space Basic Runtime Modes
Embedded space
A space instance that runs within the application memory address space
Accessed by reference without going through network or serialization calls
Most efficient configuration mode
Used as the primary space configuration setup
Remote Space
Accessing a remote space involves network calls and serialization/de-serialization of the objects between the client and the space process
Used in cases where:
Client application cannot run an embedded space (due to memory capacity limitations, etc.)
In cases where there are a large number of concurrent updates on the same object using different remote processes

Introduction - Space Basic Operations

Introduction - Basic Deployment Topologies
Primary-Backup
Partitioned Feeder Feeder Partitioned + Backup Feeder

Introduction Space Based Architecture – Business logic and data collocated Primary 1 Primary 2 Primary 3 Backup 3 Backup 2 Backup 1 Replication Replication Replication Pushing data into the backend system In-Memory-Data-Grid and collocated Processing units Collects results / reporting Service

The Benchmarks

Scale up Throughput Benchmark

Scale up Throughput Benchmark – Physical Deployment Topology
Remote (multiple machines , multiple processes)
white box Client X4450 GigaSpaces 4 spaces , one per GSC X4450 GigaSpaces 4 spaces , one per GSC Switched Ethernet LAN
Embedded (one machine , one process)
X4450 Client GigaSpaces 8 spaces

Scale up Throughput Benchmark – Embedded mode 20% drop up to 16 threads hitting the system with 1.5 M read/sec 1.8 Million read sec! 1.1 Million write/take sec!

Scale up Throughput Benchmark – Remote mode 20% drop up to 24 users hitting the system with 65,000 red/sec 90,00 read sec! 45,00 write/take sec!

Scale up Throughput Benchmark - Results
Embedded (excluding HA)
1.8 million read operations/sec with 20 threads
1.1 million write/take operations/sec with 20 threads
0.8 scalability ratio up to 16 threads with read operations
64 bit JVM , Java 1.6 Performance version, 10 G Heap Size
Up to 10 threads: ~100K read/sec , ~80K write-take/sec per thread
Remote (including HA)
90,000 read operations/sec with 50 threads
46,000 write/take operations/sec with 50 threads
0.8 scalability ratio up to 24 threads
0.6 scalability ratio up to 42 threads
Both Client running on white box, spaces running on x4450
Space: 64 bit JVM , 3 G Heap Size, Client: 64 bit JVM , 2 G Heap Size

Architecture
Data-Grid running on x4450 (2 servers – 48 cores)
Data-Grid Size – 4 partitions with/out backup
Client running on white box (1 server – 24 cores)
Multi threaded application performs write operations in stead rate (1000 write/sec)
Client Latency Measured

Scale up Latency Benchmark– Physical Deployment Topology white box Client X4450 GigaSpaces 4 spaces , one per GSC X4450 GigaSpaces 4 spaces , one per GSC Switched Ethernet LAN

Scale up Latency Benchmark Less than 20% drop up to 20 users hitting the system with 20,000 write/sec

Scale up Latency Benchmark - Results
<1 millisecond latency for 20 users running with 1000 write operations/sec including HA.
<0.4 millisecond latency for 20 users running with 1000 write operations/sec excluding HA.
True linear Scalability Ratio up to 8 users (8K write/sec)
>0.8 Scalability Ratio for 22 users (22K write/sec)
All the results above were taken with:
Total of 20,000 operations/sec
4 K object payload
1gE network
ping latency 0.1 ms.
The test object had 8 fields where 3 of them are indexed (4 String Fields , 2 Long fields, 2 Integer fields). The test Class did not implemented Externalizable and did not had any special truing or optimization to deuce its footprint or speedup its serialization.

Web Application Benchmark

Web Application Benchmark – The Pet Clinic
The pet Clinic application is popular example for on-line applications implementations over the web.
The original application has been modified to use GigaSpaces as the system of record instead of a database.

Architecture – Step 1- Request Submission Get request and invoke Service Data Grid Step 1 - Latency

Architecture – Step 2- Retrieve Results Page Generation Data Grid Latency measured: Step 1 + 2 Step 2 - Latency

Web Application Benchmark – Physical Deployment Topology X4150 Apache Load Balancer mySQL X4450 GigaSpaces 4 spaces Web servers ,Services X4450 GigaSpaces 4 spaces Web servers ,Services Switched Ethernet LAN white box Client JMeter Switched Ethernet LAN

Web Application Benchmark Results – Latency , Scalability Only 20% drop up to 20 users hitting the system with 7000 requests/sec having 2.8 ms latency

Web Application Benchmark Results - Capacity The Users factor is 50 - Every LAN based user equals 50s WAN based users due-to the inherit latency of the internet (Min latency over the WAN 100ms , over the LAN 2ms)

Web Application Benchmark - Results
Architecture
Data-Grid and Web-Servers running on x4450 (2 servers – 48 cores)
Data-Grid Size – 2 partitions with backup
Client (using JMeter) running on white box (1 server – 24 cores)
Database and Apache Load Balancer running on 4150 (1 server - 4 cores)
>6 millisecond latency for 100 users over the LAN with 3 web servers.
>0.8 Scalability Ratio for 20 users with 3 web servers
>0.6 Scalability Ratio for 50 users with 3 web servers
Capacity – 16223 page generation/sec with 3 web servers

Risk Calculation Benchmark

Risk Calculation Benchmark Workers Data-Grid Submit Calculation Request

Risk Calculation Benchmark – Physical Deployment Topology X4450 GigaSpaces spaces Workers ,Services X4450 GigaSpaces spaces Workers ,Services X4150 Client Switched Ethernet LAN White box GigaSpaces spaces Workers ,Services

Risk Calculation Benchmark Results – Calc Time , Scalability Less than 20% drop up to 32 workers

Risk Calculation Benchmark Results
Architecture
Client: x4150 – 4 cores 3GHz
Data Grid and Workers – 3 x4450 machines with 24 cores 2.67 GHz
Data-Grid size – 8 partitions
Business logic
Calculating 4096 Credit Risks using Monte Carlo simulation
>0.8 Scalability Ratio for 32 workers
>0.6 Scalability Ratio for 64 workers

Special Tuning
OS Tuning
Add the following line to the /etc/system file:
set idle_cpu_prefer_mwait=0
JVM Tuning
6G Xmx and Xmx for the embedded space benchmark
-XX:LargePageSizeInBytes=256m
-XX:ParallelGCThreads=8
GigaSpaces Tuning
com.gs.transport_protocol.lrmi.selector.threads=8
org.jini.rio.monitor.provisioningPoolMaxThreads = 32
mySQL Tuning
None
Apache Load-Balancer Tuning
None

Comparing 6 cores CPU with 4 cores CPU

Comparing 6 core CPU with 4 Core CPU
We run the throughput benchmarks on 4 cores CPU
Embedded space throughput
Remote space throughput
The 4 cores CPU clock was 10% faster than the 6 cores CPU
4 cores CPU clock speed 2.93GHz
6 cores CPU clock speed 2.66GHz
The theoretical performance boost of the 6 cores CPU should be 45%
The results where ~30% performance boost

Comparison of 6 with 4 Core CPU (16 vs. 24 cores) Embedded mode 20% difference with 16 threads 300,000 extra read oper/sec with 24 cores! 180,000 extra write/take oper/sec with 24 cores!

Comparing 6 core CPU with 4 Core CPU (16 vs. 24 cores) Embedded mode Better throughput with 24 cores Better Scalability with 24 cores – 24-29% better

Comparing 6 core CPU with 4 Core CPU (16 vs. 24 cores) Remote mode ~30% difference with 24 threads 10,000 extra read oper/sec with 24 cores! 20,000 extra write/take oper/sec with 24 cores!

Comparing 6 core CPU with 4 Core CPU (16 vs. 24 cores) Remote mode Better throughput with 24 cores Better Scalability with 24 cores 12-33 % better

Conclusions
The Sunx4450-Intel 7460-GigaSpaces XAP Platform is a very scalable platform providing both low latency and high throughput
The six cores CPU provided ~30% better scalability and performance compared to 4 cores CPU
Fits both On-line and HPC based systems
Having 4 fast CPUs with 6 cores each allows applications to scale up with minimal affect on performance and latency. Using GigaSpaces:
Scaling out can be done with minimal impact on the latency and deployment topology.
Scaling out can be done while system is running
Solaris 10 OS and JVM 6 are very stable and provide deterministic and predictable environment – even for the most demanding scenarios.
GigaSpaces In-Memory-Data-Grid provides locality of reference avoiding the need to access the database for each business transaction.
GigaSpaces as a scale out application server allows very complex applications to be deployed very easily , and virtualizes the actual deployment model from the physical network and machine topology.
Support failover , auto recovery and auto seal-healing
Dynamic SLA based scalability

Special Tuning
Network Tuning
1. The automatic interrupts rate adjustment in the Solaris IP network stack was recommended to be disabled by adding the following entry in the /etc/system file: set dld:dld_opt=2 The dld:dld_opt parameter is used to control interrupt scheduling within the IP network stack. By default, the IP code will try to automatically adjust the rate of interrupts with the intention to smooth the rate and reduce delays for low and moderate network traffic. At high interrupt rates, automatic adjustment is often counterproductive, causing more work in the IP stack which is not needed. Automatic adjustment will not be done by setting the option to the value “2”. 2. For the 1GbE network infrastructure, it is recommended to disable the interrupt throttling on the e1000g interfaces by adding the following two entries in the /kernel/srv/e1000g.conf file: intr_adaptive=0,0,0,0,0,0,0,0; intr_throttling_rate=0,0,0,0,0,0,0,0; The intr_adaptive and intr_throttling_rate variables define how interrupt blanking (coalescing) is implemented by the driver for each NIC that it controls. Each element inthe list is the value for a specific NIC from e1000g0 to e1000g15 (higher numbered NICs will likely not be present). The table examples show 16 possible NICs being changed, which is not necessary in almost all cases. If interrupt blanking is disabled, packets are processed by the driver as soon as they arrive. If interrupt blanking is enabled, packets are processed by the driver when the interrupt is issued. Enabling blanking allows the network stack to delay processing single packets with the assumption that additional packets will follow shortly which can then all be processed with a single interrupt. This will add latency, but will reduce CPU utilization by reducing the number of interrupts that will need to be handled. The variable intr_adaptive disables (0) or enables (1) the interrupt blanking mechanism provided by the Solaris Generic LAN Driver (GLD) framework. When this tunable is disabled, the intr_throttling_rate parameter is available to configure interrupt blanking manually. The variable intr_throttling_rate (0 - 65535) specifies the inter-interrupt delay to realize the interrupt blanking (coalescing). The intr_throttling_rate variable takes effect only when intr_adaptive is disabled. Smaller values of intr_throttling_rate mean higher interrupt rates/less coalescing; larger values mean lower interrupt rates/more coalescing. The value 0 indicates that there is no interrupt coalescing - an interrupt is fired immediately when a packet is received. It is recommended that for systems handling lower message rates, that the defaults (intr_adaptive set to 1, which disables intr_throttling_rate) be used. Systems that handle very high loads that are relatively constant or which are very latency sensitive should set intr_adaptive and intr_throttling_rate to 0 only for the NIC(s) carrying that load.

Need help?
Email us at [email_address]
Thank You!

Sunx4450 Intel7460 GigaSpaces XAP Platform Benchmark

by shayhassidim

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 9

Statistics

Sunx4450 Intel7460 GigaSpaces XAP Platform Benchmark Presentation Transcript

http://www.slideshare.net	5
http://jisi.dreamblog.jp	3
http://www.linkedin.com	1