PostgreSQL Portland Performance Practice Project - Database Test 2 Howto

PostgreSQL Portland Performance Practice Project Database Test 2 (DBT-2) How to Use the Kit Mark Wong markwkm@postgresql.org Portland State University March 12, 2009

How to use the DBT-2 test kit Further questions regarding the test kit can be sent to the mailing list: osdldbt-general@lists.sourceforge.net You can sign up for the mailing list here: http://lists.sourceforge.net/mailman/listinfo/osdldbt-general

WARNING This test kit is a work in progress. The last “release” was February 11, 2007. The next “release” still needs a bit of work before it’s polished. So let’s work with the source repository and hope it still works with these instructions.

Prerequisites for building PostgreSQL must be installed. ◮ PostgreSQL executables must be in your path. ◮ cmake must be installed, used to build the kit. ◮ A C compiler. ◮

Recommended for running None of these are required for running, but help characterize different aspects of the system. Test::Parser perl module for calculating test results. ◮ systat, procps, and iopp for collecting system statistics. ◮ readprofile for Linux kernel profiling. ◮ oprofile for additional Linux kernel profiling, and user space ◮ profiling, call graphs, and source code and assembly annotations.

Prerequisites for analyzing gnuplot to for making charts. ◮ Test::Presenter perl module for the modules that produce ◮ the charts.

Get the kit git clone git://git.postgresql.org/git/~markwkm/dbt2.git git is not required, you can download a tarball by clicking on the snapshot link from here: http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary

Relevant documentation Documentation to supplement or contradict what is in this howto presentation. Files to note: INSTALL ◮ README ◮ README-POSTGRESQL ◮ WARNING: Documentation may be incorrect.

Build and Install the Kit cmake CMakeLists.txt make Note: The CMakeLists.txt ﬁle is not very polished. The kit will not build properly if both MySQL and PostgreSQL are in your path. make install DESTDIR=/usr/local Note: Don’t fear, you can uninstall the kit: xargs rm < install_manifest.txt

Install supporting perl modules Install from cpan: cpan install Test::Parser cpan install Test::Presenter Install from source1 : svn co https://svn.sourceforge.net/svnroot/crucible/Test-Parser cd Test-Parser perl Makefile.PL make make install svn co https://svn.sourceforge.net/svnroot/crucible/Test-Presenter cd Test-Presenter perl Makefile.PL make make install 1 You can use subversion or git to check out the source code.

Build the C stored functions cd storedproc/pgsql/c make make install Note: The make install command needs to be run as the owner of the PostgreSQL installation, and the PostgreSQL installation to be used must in the path.

Setting up the user environment export DBNAME=dbt2 # Defines the database name to use. export PGDATA=/var/lib/data # Defines where the # database will be built. export DBT2TSDIR=/mnt/dbt2 # Defines where additional # mount points are for use # with tables spaces.

Build the database Build a 1 warehouse database, generating the load file in /tmp/datafile: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 Again but use the -b flag to load the tables and build the indexes in parallel: /tmp/datafile. dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b Yet again but use the -t flag to use the predefined location specified to use tablespaces: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b -t Note: If you have a database built already, the -r flag is need to drop the database first. Also if data files are already created, drop the use of the -g flag so they are not generated again.

Tablespace mount points for tables In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points: customer - The customer table. ◮ district - The district table. ◮ history - The history table. ◮ item - The item table. ◮ new order - The new order table. ◮ order line - The order line table. ◮ orders - The orders table. ◮ stock - The stock table. ◮ warehouse - The warehouse table. ◮

Tablespace mount points for indexes In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points for the primary key indexes: pk customer - The customer table primary key index. ◮ pk district - The district table primary key index. ◮ pk item - The item table primary key index. ◮ pk new order - The new order table primary key index. ◮ pk order line - The order line table primary key index. ◮ pk orders - The orders table primary key index. ◮ pk stock - The stock table primary key index. ◮ pk warehouse - The warehouse table primary key index. ◮ For the rest of the indexes: index1 - An index on the orders table. ◮ index2 - An index on the customer table. ◮

Run a test Run a 2 minute test (120 seconds) against a 1 warehouse database and save the test results to /tmp/test. dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/test

Results! If the Test::Parser perl module is install and the test ran successfully you’ll see results formatted something like: Response Time (s) Transaction % Average : 90th % Total Rollbacks % ------------ ----- --------------------- ----------- --------------- ----- Delivery 3.97 11.886 : 13.391 44959 0 0.00 New Order 45.34 10.673 : 11.887 513109 5054 0.99 Order Status 4.01 10.408 : 11.667 45409 0 0.00 Payment 42.66 10.396 : 11.688 482790 0 0.00 Stock Level 4.01 10.278 : 11.539 45352 0 0.00 ------------ ----- --------------------- ----------- --------------- ----- 8482.31 new-order transactions per minute (NOTPM) 59.3 minute duration 0 total unknown errors 1041 second(s) ramping up

Generating a report Generate a report with charts2 : dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report An index.html ﬁle will be created in /tmp/test/report/index.html 2 Test::Presenter and gnuplot must be installed.

Generating a report with tablespaces To determine how to name your devices in the comma separated list for each ﬂag, look to see how the devices are named by iostat. For example: dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report --log cciss/c0d4,cciss/c0d5 Transaction log device: --log ◮

Flags for plotting speciﬁc i/o data for tables Table devices: --customer ◮ --district ◮ --history ◮ --item ◮ --new order ◮ --order line ◮ --orders ◮ --stock ◮ --warehouse ◮

Flags for plotting speciﬁc i/o data for indexes Primary key index devices: --pkcustomer ◮ --pkdistrict ◮ --pkitem ◮ --pknew order ◮ --pkorder line ◮ --pkorders ◮ --pkstock ◮ --pkwarehouse ◮ Other indexes device: --index1 ◮ --index2 ◮

Materials Are Freely Available PDF http://www.slideshare.net/markwkm ◮ LTEX Beamer (source) A ◮ http://git.postgresql.org/?p=~ markwkm/performance-tuning.git

Time and Location When: 2nd Thursday of the month Location: Portland State University Room: FAB 86-01 (Fourth Avenue Building) Map: http://www.pdx.edu/map.html

Coming up next time. . . Linux ﬁle system testing using ﬁo3 . __ __ / ~~~/ . o O ( Thank you! ) ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot; 3 http://brick.kernel.dk/snaps/

Acknowledgements Haley Jane Wakenshaw __ __ / ~~~/ ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot;

License This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, (a) visit http://creativecommons.org/licenses/by/3.0/us/; or, (b) send a letter to Creative Commons, 171 2nd Street, Suite 300, San Francisco, California, 94105, USA.

http://ivanov.wordpress.com	33
http://www.slideshare.net	22
http://grosser:8080	3

PostgreSQL Portland Performance Practice Project - Database Test 2 Howto

by Mark Wong on Mar 05, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 58

Statistics

PostgreSQL Portland Performance Practice Project - Database Test 2 Howto — Presentation Transcript