Free Load Testing Tools for Oracle Database – Which One Do I Use?

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
Free Load Testing Tools for Oracle
Database – Which One Do I Use?
Christian Antognini
8 November 2016

@ChrisAntognini
Free Load Testing Tools for Oracle Database – Which One Do I Use?2 08/11/2016
Senior principal consultant, trainer and partner at Trivadis
christian.antognini@trivadis.com
http://antognini.ch
Focus: get the most out of Oracle Database
Logical and physical database design
Query optimizer
Application performance management
Author of Troubleshooting Oracle Performance (Apress, 2008/14)
OakTable Network, Oracle ACE Director

Agenda
1. Introduction
2. Free Load Testing Tools I Use
3. Summary

Introduction

Common Situations Leading to Database Load
Generation
Perform usability and functionality checks of tools or utilities that require a
non-trivial load to be carried out
Assess the performance of a (new) platform or storage subsystem
Verify whether a set of SQL statements executed on a specific environment
and/or configuration fulfils the expected performance requirements

Golden Rule
Use the right tool
for the right job!

Free Load Testing Tools in My Toolbox
Swingbench
The Silly Little Oracle Benchmark (SLOB)
Apache JMeter

Swingbench

What’s Swingbench?
Load generator designed to stress test Oracle Database
Includes four configurable synthetic benchmarks
OrderEntry, SalesHistory, CallingCircle and StressTest
Extensible with user-defined transactions (StressTest provides an example)
Consists of several CLI and GUI utilities
Developed by Dominic Giles
Available at http://www.dominicgiles.com/swingbench.html

What Can Swingbench Do?
Carry out, with minimal effort, all necessary operations to run one of the four
provided benchmarks
Create test schema (incl. data population)
Generate load
Store key performance indicators about test run

Swingbench Key Components – Setup
Three wizards, that also expose a CLI, are available: OrderEntry, SalesHistory
and CallingCircle
StressTest doesn’t have a wizard because it requires no setup
XML file config dataoewizard
shwizard
ccwizard

Swingbench Key Components – Load Generation
swingbench
minibench
charbench
XML file
XML file
results2pdf
PDF file
config
load KPI

Swingbench: Demo

Swingbench: Example of Utilization (1)
Purpose
Observe the impact of a vMotion
started while a database server
is subject to a non-trivial load
Benchmark
OrderEntry, scale 10
900 transactions/second

0
20
40
60
80
13:58:08
13:58:18
13:58:28
13:58:38
13:58:48
13:58:58
13:59:08
13:59:18
13:59:28
13:59:38
13:59:48
13:59:58
14:00:08
14:00:18
14:00:28
14:00:38
14:00:48
14:01:02
14:01:13
14:01:23
14:01:33
14:01:43
14:01:53
14:02:03
14:02:13
14:02:23
14:02:33
AAS
Time
vMotion took 79 seconds

Purpose
Quantify the overhead of TDE
Compare encryption algorithms
Benchmark
OrderEntry, scale 50
160 distinct tests (30 minutes each)
– Users: 10..100
– Encryption: None, AES128,
AES192, AES256
– With and without AES-NI
– Buffer cache: 7GB and 70GB

0
500
1000
1500
2000
2500
No Yes
Transactions/Second
AES-NI
AES128
AES192
AES256
None
0
2
4
6
8
10
12
14
No Yes
CPU/Transaction[ms]
AES-NI
AES128
AES192
AES256
None

The Silly Little Oracle Benchmark
(SLOB)

What’s SLOB?
Platform performance and stability-testing framework that
makes Oracle Database execute SQL statements
Synthetic benchmark
Consists of a few scripts
Developed by Kevin Closson
Available at https://kevinclosson.net/slob/

What Can SLOB Do?
Create one or several test schemas (incl. data population)
Generate load
Random PIO and/or LIO
– Generation of sequential/large PIO by forcing a FTS (e.g. invisible index)
Read only or read write
Lite or heavy redo generation
Store DB as well as OS key performance indicators about test run

SLOB Key Components – Setup
slob.conf config data
setup.sh

SLOB Key Components – Load Generation
runit.sh
slob.conf
AWR
reports
iostat
mpstat
vmstat
awr_info.sh
text file
config
load

SLOB: Demo

SLOB: Example of Utilization (1)
Purpose
Find out the maximum number
of random IOPS (and their
average latency) that a storage
subsystem can sustain
Benchmark
100GB
Read-only PIO/LIO

72563
93546
103718 103299 101258
48 64 96 128 160
PhysicalReadsperSecond
Sessions
2.9 2.8 4.0 5.2 6.3
31 33
44
60
78
48 64 96 128 160
LatencyinMilliseconds
Sessions
db file sequential reads db file parallel reads

Purpose
Assess disk I/O performance in
a VMware environment
Questions
– What’s the impact of running
several DB per VM?
– What’s the impact of running
several VM per server?
Benchmark
100GB
1/2/4/8 VM
One/several DB per VM
Each server running an increasing
number of sessions
Read-only and read-write tests

Apache JMeter

What’s JMeter?
Application designed to load test functional behavior and measure
performance
No out-of-the box benchmark available  DIY
Consists of several CLI and GUI utilities
Open source project supported by a number of contributors
Available at http://jmeter.apache.org/

What Can JMeter Do?
Many things!
For the intent of this presentation:
Execute SQL statements
through JDBC
Verify whether the executions
return the expected output
and/or in the expected time
Store key performance
indicators
But it’s “only” a framework… it requires
time to “code” a benchmark
Use real schema with data
Use the SQL statements to be
verified
– Bind variables can be either
generated or provided as input

SLOB Key Components – Load Generation
front end
GUI/CLI
test plan
user
defined
output
text file
config
load
KPI
log

JMeter: Demo

JMeter: Example of Utilization (1)
Purpose
Verify that a set of queries
performs acceptably under load
Verify that the DB server
supports the expected load
Benchmark
33 distinct queries, each with each
own set of bind variable values,
randomly picked out
100 queries/minute

0
500
1000
1500
2000
2500
3000
3500
4000
407
408
503
114
303
111
115
112
308
101
301
117
204
113
118
105
306
116
505
120
201
504
304
305
307
404
403
401
501
203
202
502
302
AverageElapsedTime[ms]
Query

AAS during a period of 13 minutes

Summary

Comparison
Swingbench SLOB JMeter
Can generates load for
Oracle
Database
Oracle
Database
any DB with
JDBC support
Synthetic benchmark  
User-defined benchmark () 
Effort to implement minimal minimal low  high
GUI  
CLI for automation with scripts   
Distributed load generation  

Tool of Choice for Common Situations Leading to
Database Load Generation
Perform usability and functionality checks of tools or utilities that require a
non-trivial load to be carried out
 Swingbench
Assess the performance of a (new) platform or storage subsystem
 SLOB
Verify whether a set of SQL statements executed on a specific environment
and/or configuration fulfils the expected performance requirements
 Apache JMeter

Summary
No single tool can cover all
requirements
If used correctly, Swingbench and
SLOB are simple and effective tools
To run a specific set of SQL
statements don’t spend time build
your own script/tool, use JMeter

Questions and Answers
Christian Antognini
Senior Principal Consultant
christian.antognini@trivadis.com
@ChrisAntognini
08/11/2016 Free Load Testing Tools for Oracle Database – Which One Do I Use?44

Free Load Testing Tools for Oracle Database – Which One Do I Use?

More Related Content

What's hot

Similar to Free Load Testing Tools for Oracle Database – Which One Do I Use?

More from Christian Antognini

Recently uploaded

Free Load Testing Tools for Oracle Database – Which One Do I Use?