Discussion on PostgreSQL and SQL Server query optimisation, including indexes and examining query plans.

1. Query Optimisation Troubleshooting long running queries and indexing issues Thursday May 20th, 2010

2. Overview database systems are designed to cope with tables in excess of hundreds of millions of rows over time, tables bloat – need to continually assess or index proactively different terminology between PostgreSQL and SQL Server – methods are fundamentally the same Indexes provide pointers to rows or ranges of data in a table FILLFACTOR means space to move rows around on the data pages

3. Index Terminology

4. Tools to help you

5. Finding the slow query nHibernate and others cause “problems” – only if tables and indexes aren’t updated to reflect what nHibernate will do In PostgreSQL, sometimes difficult to trap the query causing the problem - can set the log_min_duration in the log to trap long running queries In SQL, can use profiler in real time to catch the query Transactions – any open?

6. PostgreSQL – Graphical EXPLAIN

7. SQL Server – Estimated/Actual Plan

8. Not the worst… SELECT this_.id as id172_4_, this_.email as email172_4_, this_.postcode as postcode172_4_, this_.annual_mileage as annual4_172_4_, this_.created_at as created5_172_4_, this_.modified_at as modified6_172_4_, this_.vehicle_year as vehicle7_172_4_, this_.years_no_claims as years8_172_4_, this_.no_claims_protected as no9_172_4_, this_.vehicle_value as vehicle10_172_4_, this_.voluntary_excess as voluntary11_172_4_, this_.is_completed as is12_172_4_, this_.is_callcentre as is13_172_4_, this_.renewal_date as renewal14_172_4_, this_.vehicle_registration as vehicle15_172_4_, this_.policy_start_date as policy16_172_4_, this_.cap_id as cap17_172_4_, this_.cover_type_id as cover18_172_4_, this_.overnight_location_id as overnight19_172_4_, this_.vehicle_usage_id as vehicle20_172_4_, this_.access_point_id as access21_172_4_, this_.status_id as status22_172_4_, paymentdet3_.id as id164_0_, paymentdet3_.quote_id as quote2_164_0_, paymentdet3_.account_name as account3_164_0_, paymentdet3_.account_number as account4_164_0_, paymentdet3_.sort_code as sort5_164_0_, paymentdet3_.bank_name as bank6_164_0_, paymentdet3_.branch as branch164_0_, paymentdet3_.charge_percentage as charge8_164_0_, paymentdet3_.number_of_installments as number9_164_0_, paymentdet3_.start_date as start10_164_0_, paymentdet3_.renewal_date as renewal11_164_0_, paymentdet3_.bank_address_id as bank12_164_0_, paymentdet3_.loan_amount as loan13_164_0_, paymentdet3_.deposit as deposit164_0_, paymentdet3_.installment_amount as install15_164_0_, personalde4_.id as id167_1_, personalde4_.telephone as telephone167_1_, personalde4_.quote_id as quote3_167_1_, personalde4_.address_id as address4_167_1_, covernoten5_.id as id76_2_, covernoten5_.quote_id as quote2_76_2_, covernoten5_.campaign_id as campaign3_76_2_, covernoten5_.sequence_number as sequence4_76_2_, d1_.id as id86_3_, d1_.forename as forename86_3_, d1_.surname as surname86_3_, d1_.date_of_birth as date4_86_3_, d1_.is_female as is5_86_3_, d1_.length_of_licence as length6_86_3_, d1_.accidents_count as accidents7_86_3_, d1_.ordinal as ordinal86_3_, d1_.employers_business_id as employers9_86_3_, d1_.occupation_id as occupation10_86_3_, d1_.quote_id as quote11_86_3_, d1_.licence_type_id as licence12_86_3_, d1_.title_id as title13_86_3_ FROM quotes this_ left outer join payment_details paymentdet3_ on this_.id=paymentdet3_.quote_id left outer join personal_details personalde4_ on this_.id=personalde4_.quote_id left outer join covernote_numbers covernoten5_ on this_.id=covernoten5_.quote_id inner join drivers d1_ on this_.id=d1_.quote_id WHERE d1_.surname ilike'test%' and this_.id in ( SELECT distinct this_0_.id as y0_ FROM quotes this_0_ inner join campaign_quotes campaignqu1_ on this_0_.id=campaignqu1_.quote_id inner join campaigns campaign2_ on campaignqu1_.campaign_id=campaign2_.id WHERE this_0_.modified_at between '01/01/2010 00:00:00' and '15/01/2010 00:00:00' and (this_0_.status_id = 1 or campaignqu1_.selected_quote = True) and campaign2_.is_drive_away = True) ORDER BY this_.modified_atdesc

9. EXPLAIN - Query Plan

10. Lab 1 – troubleshooting in PostgreSQL

11. Lab 2 – troubleshooting SQL Server

12. When indexes don’t work…. each database system uses what’s called an “optimiser” Factors influencing the optimiser’s choice of execution plan: statistics trivial plan match caching strategies available indexes Spread of data is important

13. Further Reading http://developer.postgresql.org/pgdocs/postgres/indexes-examine.html http://wiki.postgresql.org/wiki/Image:Explaining_EXPLAIN.pdf http://www.simple-talk.com/sql/learn-sql-server/sql-server-index-basics/ http://www.vbforums.com/showthread.php?t=361513 – when to use index hints in sql server http://www.mssqltips.com/tip.asp?tip=1206 – understanding sql server indexing