PostgreSQL Portland Performance Practice Project - Database Test 2 Howto
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

PostgreSQL Portland Performance Practice Project - Database Test 2 Howto

  • 7,668 views
Uploaded on

Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction......

Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,668
On Slideshare
7,581
From Embeds
87
Number of Embeds
5

Actions

Shares
Downloads
110
Comments
0
Likes
3

Embeds 87

http://ivanov.wordpress.com 60
http://www.slideshare.net 22
http://grosser:8080 3
https://ivanov.wordpress.com 1
http://www.slidesearchengine.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PostgreSQL Portland Performance Practice Project Database Test 2 (DBT-2) How to Use the Kit Mark Wong markwkm@postgresql.org Portland State University March 12, 2009
  • 2. How to use the DBT-2 test kit Further questions regarding the test kit can be sent to the mailing list: osdldbt-general@lists.sourceforge.net You can sign up for the mailing list here: http://lists.sourceforge.net/mailman/listinfo/osdldbt-general
  • 3. WARNING This test kit is a work in progress. The last “release” was February 11, 2007. The next “release” still needs a bit of work before it’s polished. So let’s work with the source repository and hope it still works with these instructions.
  • 4. Prerequisites for building PostgreSQL must be installed. ◮ PostgreSQL executables must be in your path. ◮ cmake must be installed, used to build the kit. ◮ A C compiler. ◮
  • 5. Recommended for running None of these are required for running, but help characterize different aspects of the system. Test::Parser perl module for calculating test results. ◮ systat, procps, and iopp for collecting system statistics. ◮ readprofile for Linux kernel profiling. ◮ oprofile for additional Linux kernel profiling, and user space ◮ profiling, call graphs, and source code and assembly annotations.
  • 6. Prerequisites for analyzing gnuplot to for making charts. ◮ Test::Presenter perl module for the modules that produce ◮ the charts.
  • 7. Get the kit git clone git://git.postgresql.org/git/~markwkm/dbt2.git git is not required, you can download a tarball by clicking on the snapshot link from here: http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary
  • 8. Relevant documentation Documentation to supplement or contradict what is in this howto presentation. Files to note: INSTALL ◮ README ◮ README-POSTGRESQL ◮ WARNING: Documentation may be incorrect.
  • 9. Build and Install the Kit cmake CMakeLists.txt make Note: The CMakeLists.txt file is not very polished. The kit will not build properly if both MySQL and PostgreSQL are in your path. make install DESTDIR=/usr/local Note: Don’t fear, you can uninstall the kit: xargs rm < install_manifest.txt
  • 10. Install supporting perl modules Install from cpan: cpan install Test::Parser cpan install Test::Presenter Install from source1 : svn co https://svn.sourceforge.net/svnroot/crucible/Test-Parser cd Test-Parser perl Makefile.PL make make install svn co https://svn.sourceforge.net/svnroot/crucible/Test-Presenter cd Test-Presenter perl Makefile.PL make make install 1 You can use subversion or git to check out the source code.
  • 11. Build the C stored functions cd storedproc/pgsql/c make make install Note: The make install command needs to be run as the owner of the PostgreSQL installation, and the PostgreSQL installation to be used must in the path.
  • 12. Setting up the user environment export DBNAME=dbt2 # Defines the database name to use. export PGDATA=/var/lib/data # Defines where the # database will be built. export DBT2TSDIR=/mnt/dbt2 # Defines where additional # mount points are for use # with tables spaces.
  • 13. Build the database Build a 1 warehouse database, generating the load file in /tmp/datafile: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 Again but use the -b flag to load the tables and build the indexes in parallel: /tmp/datafile. dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b Yet again but use the -t flag to use the predefined location specified to use tablespaces: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b -t Note: If you have a database built already, the -r flag is need to drop the database first. Also if data files are already created, drop the use of the -g flag so they are not generated again.
  • 14. Tablespace mount points for tables In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points: customer - The customer table. ◮ district - The district table. ◮ history - The history table. ◮ item - The item table. ◮ new order - The new order table. ◮ order line - The order line table. ◮ orders - The orders table. ◮ stock - The stock table. ◮ warehouse - The warehouse table. ◮
  • 15. Tablespace mount points for indexes In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points for the primary key indexes: pk customer - The customer table primary key index. ◮ pk district - The district table primary key index. ◮ pk item - The item table primary key index. ◮ pk new order - The new order table primary key index. ◮ pk order line - The order line table primary key index. ◮ pk orders - The orders table primary key index. ◮ pk stock - The stock table primary key index. ◮ pk warehouse - The warehouse table primary key index. ◮ For the rest of the indexes: index1 - An index on the orders table. ◮ index2 - An index on the customer table. ◮
  • 16. Run a test Run a 2 minute test (120 seconds) against a 1 warehouse database and save the test results to /tmp/test. dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/test
  • 17. Results! If the Test::Parser perl module is install and the test ran successfully you’ll see results formatted something like: Response Time (s) Transaction % Average : 90th % Total Rollbacks % ------------ ----- --------------------- ----------- --------------- ----- Delivery 3.97 11.886 : 13.391 44959 0 0.00 New Order 45.34 10.673 : 11.887 513109 5054 0.99 Order Status 4.01 10.408 : 11.667 45409 0 0.00 Payment 42.66 10.396 : 11.688 482790 0 0.00 Stock Level 4.01 10.278 : 11.539 45352 0 0.00 ------------ ----- --------------------- ----------- --------------- ----- 8482.31 new-order transactions per minute (NOTPM) 59.3 minute duration 0 total unknown errors 1041 second(s) ramping up
  • 18. Generating a report Generate a report with charts2 : dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report An index.html file will be created in /tmp/test/report/index.html 2 Test::Presenter and gnuplot must be installed.
  • 19. Generating a report with tablespaces To determine how to name your devices in the comma separated list for each flag, look to see how the devices are named by iostat. For example: dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report --log cciss/c0d4,cciss/c0d5 Transaction log device: --log ◮
  • 20. Flags for plotting specific i/o data for tables Table devices: --customer ◮ --district ◮ --history ◮ --item ◮ --new order ◮ --order line ◮ --orders ◮ --stock ◮ --warehouse ◮
  • 21. Flags for plotting specific i/o data for indexes Primary key index devices: --pkcustomer ◮ --pkdistrict ◮ --pkitem ◮ --pknew order ◮ --pkorder line ◮ --pkorders ◮ --pkstock ◮ --pkwarehouse ◮ Other indexes device: --index1 ◮ --index2 ◮
  • 22. Materials Are Freely Available PDF http://www.slideshare.net/markwkm ◮ LTEX Beamer (source) A ◮ http://git.postgresql.org/?p=~ markwkm/performance-tuning.git
  • 23. Time and Location When: 2nd Thursday of the month Location: Portland State University Room: FAB 86-01 (Fourth Avenue Building) Map: http://www.pdx.edu/map.html
  • 24. Coming up next time. . . Linux file system testing using fio3 . __ __ / ~~~/ . o O ( Thank you! ) ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot; 3 http://brick.kernel.dk/snaps/
  • 25. Acknowledgements Haley Jane Wakenshaw __ __ / ~~~/ ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot;
  • 26. License This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, (a) visit http://creativecommons.org/licenses/by/3.0/us/; or, (b) send a letter to Creative Commons, 171 2nd Street, Suite 300, San Francisco, California, 94105, USA.