PostgreSQL Portland Performance Practice Project - Database Test 2 Howto

  • 4,737 views
Uploaded on

Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) …

Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,737
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
110
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PostgreSQL Portland Performance Practice Project Database Test 2 (DBT-2) How to Use the Kit Mark Wong markwkm@postgresql.org Portland State University March 12, 2009
  • 2. How to use the DBT-2 test kit Further questions regarding the test kit can be sent to the mailing list: osdldbt-general@lists.sourceforge.net You can sign up for the mailing list here: http://lists.sourceforge.net/mailman/listinfo/osdldbt-general
  • 3. WARNING This test kit is a work in progress. The last “release” was February 11, 2007. The next “release” still needs a bit of work before it’s polished. So let’s work with the source repository and hope it still works with these instructions.
  • 4. Prerequisites for building PostgreSQL must be installed. ◮ PostgreSQL executables must be in your path. ◮ cmake must be installed, used to build the kit. ◮ A C compiler. ◮
  • 5. Recommended for running None of these are required for running, but help characterize different aspects of the system. Test::Parser perl module for calculating test results. ◮ systat, procps, and iopp for collecting system statistics. ◮ readprofile for Linux kernel profiling. ◮ oprofile for additional Linux kernel profiling, and user space ◮ profiling, call graphs, and source code and assembly annotations.
  • 6. Prerequisites for analyzing gnuplot to for making charts. ◮ Test::Presenter perl module for the modules that produce ◮ the charts.
  • 7. Get the kit git clone git://git.postgresql.org/git/~markwkm/dbt2.git git is not required, you can download a tarball by clicking on the snapshot link from here: http://git.postgresql.org/?p=~markwkm/dbt2.git;a=summary
  • 8. Relevant documentation Documentation to supplement or contradict what is in this howto presentation. Files to note: INSTALL ◮ README ◮ README-POSTGRESQL ◮ WARNING: Documentation may be incorrect.
  • 9. Build and Install the Kit cmake CMakeLists.txt make Note: The CMakeLists.txt file is not very polished. The kit will not build properly if both MySQL and PostgreSQL are in your path. make install DESTDIR=/usr/local Note: Don’t fear, you can uninstall the kit: xargs rm < install_manifest.txt
  • 10. Install supporting perl modules Install from cpan: cpan install Test::Parser cpan install Test::Presenter Install from source1 : svn co https://svn.sourceforge.net/svnroot/crucible/Test-Parser cd Test-Parser perl Makefile.PL make make install svn co https://svn.sourceforge.net/svnroot/crucible/Test-Presenter cd Test-Presenter perl Makefile.PL make make install 1 You can use subversion or git to check out the source code.
  • 11. Build the C stored functions cd storedproc/pgsql/c make make install Note: The make install command needs to be run as the owner of the PostgreSQL installation, and the PostgreSQL installation to be used must in the path.
  • 12. Setting up the user environment export DBNAME=dbt2 # Defines the database name to use. export PGDATA=/var/lib/data # Defines where the # database will be built. export DBT2TSDIR=/mnt/dbt2 # Defines where additional # mount points are for use # with tables spaces.
  • 13. Build the database Build a 1 warehouse database, generating the load file in /tmp/datafile: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 Again but use the -b flag to load the tables and build the indexes in parallel: /tmp/datafile. dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b Yet again but use the -t flag to use the predefined location specified to use tablespaces: dbt2-pgsql-build-db -d /tmp/datafile -g -w 1 -b -t Note: If you have a database built already, the -r flag is need to drop the database first. Also if data files are already created, drop the use of the -g flag so they are not generated again.
  • 14. Tablespace mount points for tables In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points: customer - The customer table. ◮ district - The district table. ◮ history - The history table. ◮ item - The item table. ◮ new order - The new order table. ◮ order line - The order line table. ◮ orders - The orders table. ◮ stock - The stock table. ◮ warehouse - The warehouse table. ◮
  • 15. Tablespace mount points for indexes In the ${DBT2TSDIR} directory, disk devices can be mounted at the following points for the primary key indexes: pk customer - The customer table primary key index. ◮ pk district - The district table primary key index. ◮ pk item - The item table primary key index. ◮ pk new order - The new order table primary key index. ◮ pk order line - The order line table primary key index. ◮ pk orders - The orders table primary key index. ◮ pk stock - The stock table primary key index. ◮ pk warehouse - The warehouse table primary key index. ◮ For the rest of the indexes: index1 - An index on the orders table. ◮ index2 - An index on the customer table. ◮
  • 16. Run a test Run a 2 minute test (120 seconds) against a 1 warehouse database and save the test results to /tmp/test. dbt2-run-workload -a pgsql -d 120 -w 1 -o /tmp/test
  • 17. Results! If the Test::Parser perl module is install and the test ran successfully you’ll see results formatted something like: Response Time (s) Transaction % Average : 90th % Total Rollbacks % ------------ ----- --------------------- ----------- --------------- ----- Delivery 3.97 11.886 : 13.391 44959 0 0.00 New Order 45.34 10.673 : 11.887 513109 5054 0.99 Order Status 4.01 10.408 : 11.667 45409 0 0.00 Payment 42.66 10.396 : 11.688 482790 0 0.00 Stock Level 4.01 10.278 : 11.539 45352 0 0.00 ------------ ----- --------------------- ----------- --------------- ----- 8482.31 new-order transactions per minute (NOTPM) 59.3 minute duration 0 total unknown errors 1041 second(s) ramping up
  • 18. Generating a report Generate a report with charts2 : dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report An index.html file will be created in /tmp/test/report/index.html 2 Test::Presenter and gnuplot must be installed.
  • 19. Generating a report with tablespaces To determine how to name your devices in the comma separated list for each flag, look to see how the devices are named by iostat. For example: dbt2-generate-report --indir /tmp/test --outdir /tmp/test/report --log cciss/c0d4,cciss/c0d5 Transaction log device: --log ◮
  • 20. Flags for plotting specific i/o data for tables Table devices: --customer ◮ --district ◮ --history ◮ --item ◮ --new order ◮ --order line ◮ --orders ◮ --stock ◮ --warehouse ◮
  • 21. Flags for plotting specific i/o data for indexes Primary key index devices: --pkcustomer ◮ --pkdistrict ◮ --pkitem ◮ --pknew order ◮ --pkorder line ◮ --pkorders ◮ --pkstock ◮ --pkwarehouse ◮ Other indexes device: --index1 ◮ --index2 ◮
  • 22. Materials Are Freely Available PDF http://www.slideshare.net/markwkm ◮ LTEX Beamer (source) A ◮ http://git.postgresql.org/?p=~ markwkm/performance-tuning.git
  • 23. Time and Location When: 2nd Thursday of the month Location: Portland State University Room: FAB 86-01 (Fourth Avenue Building) Map: http://www.pdx.edu/map.html
  • 24. Coming up next time. . . Linux file system testing using fio3 . __ __ / ~~~/ . o O ( Thank you! ) ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot; 3 http://brick.kernel.dk/snaps/
  • 25. Acknowledgements Haley Jane Wakenshaw __ __ / ~~~/ ,----( oo ) / __ __/ /| ( |( ^ /___ / | |__| |__|-quot;
  • 26. License This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, (a) visit http://creativecommons.org/licenses/by/3.0/us/; or, (b) send a letter to Creative Commons, 171 2nd Street, Suite 300, San Francisco, California, 94105, USA.