Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.

1. PostgreSQL Portland
Performance Practice Project
Database Test 2 (DBT-2)
How to Use the Kit
Mark Wong
markwkm@postgresql.org
Portland State University
March 12, 2009

2. How to use the DBT-2 test kit
Further questions regarding the test kit can be sent to the mailing
list: osdldbt-general@lists.sourceforge.net
You can sign up for the mailing list here:
http://lists.sourceforge.net/mailman/listinfo/osdldbt-general

3. WARNING
This test kit is a work in progress. The last “release” was February
11, 2007. The next “release” still needs a bit of work before it’s
polished. So let’s work with the source repository and hope it still
works with these instructions.

4. Prerequisites for building
PostgreSQL must be installed.
◮
PostgreSQL executables must be in your path.
◮
cmake must be installed, used to build the kit.
◮
A C compiler.
◮

5. Recommended for running
None of these are required for running, but help characterize
different aspects of the system.
Test::Parser perl module for calculating test results.
◮
systat, procps, and iopp for collecting system statistics.
◮
readprofile for Linux kernel profiling.
◮
oprofile for additional Linux kernel profiling, and user space
◮
profiling, call graphs, and source code and assembly
annotations.

6. Prerequisites for analyzing
gnuplot to for making charts.
◮
Test::Presenter perl module for the modules that produce
◮
the charts.

7. Get the kit
git clone git://git.postgresql.org/git/~markwkm/dbt2.git
git is not required, you can download a tarball by clicking on the
snapshot link from here:
http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary

8. Relevant documentation
Documentation to supplement or contradict what is in this howto
presentation.
Files to note:
INSTALL
◮
README
◮
README-POSTGRESQL
◮
WARNING: Documentation may be incorrect.

9. Build and Install the Kit
cmake CMakeLists.txt
make
Note: The CMakeLists.txt file is not very polished. The kit will not
build properly if both MySQL and PostgreSQL are in your path.
make install DESTDIR=/usr/local
Note: Don’t fear, you can uninstall the kit:
xargs rm < install_manifest.txt

10. Install supporting perl modules
Install from cpan:
cpan install Test::Parser
cpan install Test::Presenter
Install from source1 :
svn co https://svn.sourceforge.net/svnroot/crucible/Test-Parser
cd Test-Parser
perl Makefile.PL
make
make install
svn co https://svn.sourceforge.net/svnroot/crucible/Test-Presenter
cd Test-Presenter
perl Makefile.PL
make
make install
1
You can use subversion or git to check out the source code.

11. Build the C stored functions
cd storedproc/pgsql/c
make
make install
Note: The make install command needs to be run as the owner
of the PostgreSQL installation, and the PostgreSQL installation to
be used must in the path.

12. Setting up the user environment
export DBNAME=dbt2 # Defines the database name to use.
export PGDATA=/var/lib/data # Defines where the
# database will be built.
export DBT2TSDIR=/mnt/dbt2 # Defines where additional
# mount points are for use
# with tables spaces.

13. Build the database
Build a 1 warehouse database, generating the load file in
/tmp/datafile:
dbt2-pgsql-build-db -d /tmp/datafile -g -w 1
Again but use the -b flag to load the tables and build the indexes
in parallel: /tmp/datafile.
dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b
Yet again but use the -t flag to use the predefined location
specified to use tablespaces:
dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b -t
Note: If you have a database built already, the -r flag is need to
drop the database first. Also if data files are already created, drop
the use of the -g flag so they are not generated again.

14. Tablespace mount points for tables
In the ${DBT2TSDIR} directory, disk devices can be mounted at
the following points:
customer - The customer table.
◮
district - The district table.
◮
history - The history table.
◮
item - The item table.
◮
new order - The new order table.
◮
order line - The order line table.
◮
orders - The orders table.
◮
stock - The stock table.
◮
warehouse - The warehouse table.
◮

15. Tablespace mount points for indexes
In the ${DBT2TSDIR} directory, disk devices can be mounted at
the following points for the primary key indexes:
pk customer - The customer table primary key index.
◮
pk district - The district table primary key index.
◮
pk item - The item table primary key index.
◮
pk new order - The new order table primary key index.
◮
pk order line - The order line table primary key index.
◮
pk orders - The orders table primary key index.
◮
pk stock - The stock table primary key index.
◮
pk warehouse - The warehouse table primary key index.
◮
For the rest of the indexes:
index1 - An index on the orders table.
◮
index2 - An index on the customer table.
◮

16. Run a test
Run a 2 minute test (120 seconds) against a 1 warehouse database
and save the test results to /tmp/test.
dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/test

17. Results!
If the Test::Parser perl module is install and the test ran
successfully you’ll see results formatted something like:
Response Time (s)
Transaction % Average : 90th % Total Rollbacks %
------------ ----- --------------------- ----------- --------------- -----
Delivery 3.97 11.886 : 13.391 44959 0 0.00
New Order 45.34 10.673 : 11.887 513109 5054 0.99
Order Status 4.01 10.408 : 11.667 45409 0 0.00
Payment 42.66 10.396 : 11.688 482790 0 0.00
Stock Level 4.01 10.278 : 11.539 45352 0 0.00
------------ ----- --------------------- ----------- --------------- -----
8482.31 new-order transactions per minute (NOTPM)
59.3 minute duration
0 total unknown errors
1041 second(s) ramping up

18. Generating a report
Generate a report with charts2 :
dbt2-generate-report --indir /tmp/test
--outdir /tmp/test/report
An index.html file will be created in
/tmp/test/report/index.html
2
Test::Presenter and gnuplot must be installed.

19. Generating a report with tablespaces
To determine how to name your devices in the comma separated
list for each flag, look to see how the devices are named by iostat.
For example:
dbt2-generate-report --indir /tmp/test
--outdir /tmp/test/report
--log cciss/c0d4,cciss/c0d5
Transaction log device:
--log
◮

20. Flags for plotting specific i/o data for tables
Table devices:
--customer
◮
--district
◮
--history
◮
--item
◮
--new order
◮
--order line
◮
--orders
◮
--stock
◮
--warehouse
◮

21. Flags for plotting specific i/o data for indexes
Primary key index devices:
--pkcustomer
◮
--pkdistrict
◮
--pkitem
◮
--pknew order
◮
--pkorder line
◮
--pkorders
◮
--pkstock
◮
--pkwarehouse
◮
Other indexes device:
--index1
◮
--index2
◮

22. Materials Are Freely Available
PDF
http://www.slideshare.net/markwkm
◮
LTEX Beamer (source)
A
◮ http://git.postgresql.org/?p=~ markwkm/performance-tuning.git

23. Time and Location
When: 2nd Thursday of the month
Location: Portland State University
Room: FAB 86-01 (Fourth Avenue Building)
Map: http://www.pdx.edu/map.html

24. Coming up next time. . .
Linux file system testing using fio3 .
__ __
/ ~~~/ . o O ( Thank you! )
,----( oo )
/ __ __/
/| ( |(
^ /___ / |
|__| |__|-quot;
3
http://brick.kernel.dk/snaps/

25. Acknowledgements
Haley Jane Wakenshaw
__ __
/ ~~~/
,----( oo )
/ __ __/
/| ( |(
^ /___ / |
|__| |__|-quot;

26. License
This work is licensed under a Creative Commons Attribution 3.0
Unported License. To view a copy of this license, (a) visit
http://creativecommons.org/licenses/by/3.0/us/; or, (b)
send a letter to Creative Commons, 171 2nd Street, Suite 300, San
Francisco, California, 94105, USA.