SlideShare a Scribd company logo
1 of 39
Download to read offline
Samay Sharma | RailsConf 2019
Optimizing your app by understanding
your Postgres database
Samay Sharma
Solutions Engineering Manager
RailsConf 2019 | Minneapolis | April 2019
Samay Sharma | RailsConf 20192
• Citus - Open Source Extension to
Scale out Postgres
• Manage a team of Solutions
Engineers
• Work with clients
• Fun(?) fact : Recently got married J
Samay Sharma | RailsConf 2019
Samay Sharma | RailsConf 2019
Challenge: Relating issues to the database
• Problems you’re facing from an application perspective
• Tied to database
• How do I find the root cause?
• How do I solve it?
4
Samay Sharma | RailsConf 2019
Solution: Using Postgres’ statistics
• Postgres captures, uses and exposes many statistics.
• Monitoring statistics – Statistics about system activity
• Query planner statistics – Statistics used by the planner to choose the right query plan.
• Server Administrator statistics – Statistics about replication, size of database, tables, etc.
• It’s important to know how to use them.
5
Samay Sharma | RailsConf 2019
Problem: My application is slow
• Symptoms: Application users are seeing slow query performance.
• Your monitoring tool shows you that most of the time is going in database
calls.
• Your page is making 100s of database calls, you don’t know which query
is slow.
6
Samay Sharma | RailsConf 2019
Cache hit ratio
• Gives an idea of how much of your data is coming from the PostgreSQL
buffercache.
• For transactional apps, want to keep if >95%, ideally around 99%.
7
Samay Sharma | RailsConf 2019
How do I measure?
SELECT
sum(heap_blks_read) as heap_read,
sum(heap_blks_hit) as heap_hit,
sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio
FROM
pg_statio_user_tables;
8
Samay Sharma | RailsConf 2019
pg_statio_user_tables
select * from pg_statio_user_tables where relname = 'tbl';
-[ RECORD 1 ]---+---------
relid | 19049
schemaname | public
relname | tbl
heap_blks_read | 44260
heap_blks_hit | 10162819
idx_blks_read | 31
idx_blks_hit | 30
toast_blks_read |
toast_blks_hit |
tidx_blks_read |
tidx_blks_hit |
9
Samay Sharma | RailsConf 2019
Digging further
• If cache hit ratio is low, that could mean a lot of things.
• Maybe you have a lot of bloat.
• Maybe autovacuum is not tuned well enough.
• Maybe your queries are not optimized and you are doing a lot of sequential scans.
• Maybe you have a lot of unused indexes.
• Or, you actually have less resources and you need to scale out with Citus.
10
Samay Sharma | RailsConf 2019
Pg_stat_database
select * from pg_stat_database where datname = 'citus';
-[ RECORD 1 ]--+------------------------------
datid | 16385
datname | citus
numbackends | 3
xact_commit | 1023786
xact_rollback | 8
blks_read | 45324
blks_hit | 705650570
tup_returned | 675865817
tup_fetched | 367682017
tup_inserted | 10009861
tup_updated | 1738
tup_deleted | 600
conflicts | 0
temp_files | 4
temp_bytes | 340491008
deadlocks | 0
blk_read_time | 59.44
blk_write_time | 0
stats_reset | 2019-01-03 23:07:28.206208+00
11
Samay Sharma | RailsConf 2019
Uses for pg_stat_database
• Find out rows fetched vs returned by queries to the database
• Find out the insert / update / delete ratio for your database
• Total number of transactions executed / throughput
12
Samay Sharma | RailsConf 2019
pg_stat_user_tables
select * from pg_stat_user_tables where relname = 'tbl';
-[ RECORD 1 ]-------+------------------------------
relid | 19049
schemaname | public
relname | tbl
seq_scan | 3
seq_tup_read | 10000000
idx_scan | 2
idx_tup_fetch | 20000
n_tup_ins | 10000000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 9999977
n_dead_tup | 0
n_mod_since_analyze | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2019-04-28 22:28:42.068611+00
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
13
Samay Sharma | RailsConf 2019
Uses for pg_stat_user_tables
• Approximate number of live / dead tuples (rows) in a table
• Find out if the table is an insert / update / delete heavy table.
• Find out if your table is being auto vacuumed sufficiently or not.
• Find out if most scans on the table are sequential scans or index scans.
14
Samay Sharma | RailsConf 2019
pg_stat_user_indexes
select * from pg_stat_user_indexes where relname = 'tbl';
-[ RECORD 1 ]-+-------
relid | 19049
indexrelid | 19052
schemaname | public
relname | tbl
indexrelname | i1
idx_scan | 4
idx_tup_read | 40000
idx_tup_fetch | 40000
15
Samay Sharma | RailsConf 2019
Uses for pg_stat_user_indexes
• Figuring out unused indexes
• Ratio of index rows read vs fetched
16
Samay Sharma | RailsConf 2019
Back to cache hit rate
• Understand why you had a bad hit rate
• You see database patterns which don’t align with the application – Dive deeper into them
• Is there a lot of bloat, are some tables not vacuumed often enough – Tune autovacuum
• Are you doing a lot of sequential scans - Create indexes
• Maybe you have a lot of unnecessary indexes – Delete them
17
Samay Sharma | RailsConf 2019
Generic vs specific optimizations
• Follow above practices to make generic optimizations when your
database is slow.
• How do you speed up a particular webpage?
• Or how do you find out what specific queries are slow?
18
Samay Sharma | RailsConf 2019
How do I find what queries are slow?
19
Samay Sharma | RailsConf 2019
What is `pg_stat_statements` ?
• PostgreSQL extension which tracks execution statistics of all SQL
statements executed by a server
• Normalized so similar queries grouped together
• Exposed to user w/ view named `pg_stat_statements`
20
Samay Sharma | RailsConf 2019
`pg_stat_statements` view
• Query: User_id, db_id, query_id, query text, # of times executed
• Timing: min_time, max_time, total_time, mean_time, stddev_time
• Shared blocks hit, read, written, dirtied
• Local blocks hit, read, written, dirtied
• I/O times: total time spent in reading & writing blocks.
• Enable track_io_timing to get these, otherwise will be empty
21
Samay Sharma | RailsConf 2019
Examples
• Top 10 time consuming queries
select query, total_time from pg_stat_statements ORDER BY total_time DESC limit 10
• Top 10 queries spending most time on disk reads
select query, blk_read_time from pg_stat_statements ORDER BY blk_read_time DESC limit 10
• Top 10 most common queries
select query, calls from pg_stat_statements ORDER BY calls DESC limit 10
22
Samay Sharma | RailsConf 2019
SELECT rolname,
calls,
total_time,
mean_time,
max_time,
stddev_time,
rows,
regexp_replace(query, '[ tn]+', ' ', 'g') AS query_text
FROM pg_stat_statements
JOIN pg_roles r ON r.oid = userid
WHERE calls > 100
AND rolname NOT LIKE '%backup'
ORDER BY mean_time DESC
LIMIT 15;
-[ RECORD 1] ----------------------------------------------------------------------------------------------------
- ---------------------------------------------------------------------------------------------------------------
---
rolname | citus
calls | 141
total_time | 121404.290523
mean_time | 861.023337042553
max_time | 3718.828292
stddev_time | 913.641891936023
rows | 846
query_text | SELECT ads.campaign_id, COUNT(*) FROM ads JOIN impressions i ON (ads.id = ad_id AND ads.company_id
= i.company_id) WHERE ads.campaign_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14) AND ads.company_id =
$15 AND seen_at > now()::date GROUP BY ads.campaign_id
Source: http://geekmonkey.org/2017/11/optimizing-postgresql-queries-using-pg_stat_statements-and-pg_buffercache/
Samay Sharma | RailsConf 2019
EXPLAIN ANALYZE and tune them!
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..169247.80 rows=9584 width=8) (actual time=0.641..622.851 rows=10000 loops=1)
Filter: (col1 = 1)
Rows Removed by Filter: 9990000
Planning time: 0.051 ms
Execution time: 623.185 ms
(5 rows)
24
Samay Sharma | RailsConf 2019
Interesting things about EXPLAIN
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0;
QUERY PLAN
----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..194248.69 rows=100 width=8) (actual time=0.640..630.130 rows=10000 loops=1)
Filter: ((col1 = 1) AND (col2 = 0))
Rows Removed by Filter: 9990000
Planning time: 0.072 ms
Execution time: 630.467 ms
(5 rows)
25
Samay Sharma | RailsConf 2019
Estimates vs actual
• Postgres tells you what it thinks even in the EXPLAIN plans.
• Sometimes reason for slow queries might just be that Postgres has
incorrect statistics. Notice and fix those mismatches.
• Try to ANALYZE with larger default_statistics_target.
26
Samay Sharma | RailsConf 2019
Multi-column statistics
• Postgres only captures only single column statistics.
• Sometimes, those are not enough because they can’t cover correlations.
• But, you can tell PG two columns are related.
27
Samay Sharma | RailsConf 2019
CREATE STATISTICS
CREATE STATISTICS s1 (dependencies) on col1, col2 from tbl;
ANALYZE tbl;
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..194247.76 rows=9584 width=8) (actual time=0.638..629.741 rows=10000 loops=1)
Filter: ((col1 = 1) AND (col2 = 0))
Rows Removed by Filter: 9990000
Planning time: 0.115 ms
Execution time: 630.076 ms
(5 rows)
28
Samay Sharma | RailsConf 2019
Example of impact
29
Samay Sharma | RailsConf 2019
Summary of specific query optimization
• Find the right queries to optimize using pg_stat_statements
• Use EXPLAIN ANALYZE and tune the slowest parts
• Look at differences between estimated and actual and find ways to fix
them.
30
Samay Sharma | RailsConf 2019
Bonus: Quick Estimates
31
Samay Sharma | RailsConf 2019
pg_stats
select * from pg_stats where tablename = 'tbl' and attname = 'col1';
-[ RECORD 1 ]----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------
schemaname | public
tablename | tbl
attname | col1
inherited | f
null_frac | 0
avg_width | 4
n_distinct | 1000
most_common_vals | {945,394,677,839,551,636,805,3,743,799,299,967,295,396,400,403,663,740}
most_common_freqs |
{0.00176667,0.0017,0.0016,0.0016,0.00156667,0.00156667,0.00156667,0.00153333,0.00153333,0.00153333,0.0015,0.0015,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667}
histogram_bounds |
{0,10,20,29,39,49,58,68,78,88,99,109,119,129,139,149,158,168,177,186,195,205,215,225,235,245,254,263,273,283,292,303,313,323,331,341,350,360,371,380,391,406,415,425,434,443,454,465,474,483
,492,502,511,521,531,541,550,562,571,581,591,600,610,620,630,642,652,662,672,682,692,702,712,722,731,742,754,763,773,783,792,803,813,824,834,845,855,866,876,886,897,907,917,928,938,948,958
,969,980,991,999}
correlation | 1
most_common_elems |
most_common_elem_freqs |
elem_count_histogram |
32
Samay Sharma | RailsConf 2019
Uses for pg_stats
• What is the null ratio for a particular column?
• Find the approximate number of distinct values for a column.
• What are the most common values for a column and what are the
approximate number of rows with those values?
33
Samay Sharma | RailsConf 2019
Approximate count
• Do you know how to get a very fast approximate count of rows in a table?
SELECT
relname, reltuples
FROM
pg_class
WHERE
relname = 'tbl’;
relname | reltuples
---------+-----------
tbl | 1e+07
(1 row)
34
Samay Sharma | RailsConf 2019
Summary
• Postgres exposes a lot of stats to you. In this talk, we covered mainly
performance.
• Pg_stat_* tables to find index, table, IO, vacuum statistics.
• Enable pg_stat_statements to find out your slow queries.
35
Samay Sharma | RailsConf 2019
Summary
• Use EXPLAIN ANALYZE or EXPLAIN (ANALYZE, BUFFERS) to see what
the planner is actually doing.
• Use pg_stats and pg_class to get good approximations and understand
postgres’ understanding of your data.
36
Samay Sharma | RailsConf 2019Samay Sharma | RailsConf 2019
aka.ms/citus-azure.int
Samay Sharma | RailsConf 2019
Questions?
@citusdata@samay_sharma
github.com/citusdata/citus
@microsoft
Samay Sharma | RailsConf 2019
Useful readings – Based on talk feedback
• Measuring Bloat (and more) : https://www.citusdata.com/blog/2017/10/20/monitoring-your-bloat-in-postgres/
• Queries for unused indexes and cache hit ratio : https://www.citusdata.com/blog/2017/09/29/what-performance-
can-you-expect-from-postgres/
• Tuning Autovacuum : https://www.citusdata.com/blog/2016/11/04/autovacuum-not-the-enemy/
• Multi-column statistics : https://www.citusdata.com/blog/2018/03/06/postgres-planner-and-its-usage-of-
statistics/
39

More Related Content

What's hot

Vce vxrail-customer-presentation new
Vce vxrail-customer-presentation newVce vxrail-customer-presentation new
Vce vxrail-customer-presentation newJennifer Graham
 
Things you should know about Oracle truncate
Things you should know about Oracle truncateThings you should know about Oracle truncate
Things you should know about Oracle truncateKazuhiro Takahashi
 
HKG18-318 - OpenAMP Workshop
HKG18-318 - OpenAMP WorkshopHKG18-318 - OpenAMP Workshop
HKG18-318 - OpenAMP WorkshopLinaro
 
Oracle RAC - New Generation
Oracle RAC - New GenerationOracle RAC - New Generation
Oracle RAC - New GenerationAnil Nair
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015PostgreSQL-Consulting
 
Drilling into Data with Apache Drill
Drilling into Data with Apache DrillDrilling into Data with Apache Drill
Drilling into Data with Apache DrillDataWorks Summit
 
Deploying PostgreSQL on Kubernetes
Deploying PostgreSQL on KubernetesDeploying PostgreSQL on Kubernetes
Deploying PostgreSQL on KubernetesJimmy Angelakos
 
Routed Fabrics For Ceph
Routed Fabrics For CephRouted Fabrics For Ceph
Routed Fabrics For CephShapeBlue
 
Oracle db performance tuning
Oracle db performance tuningOracle db performance tuning
Oracle db performance tuningSimon Huang
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisArnab Mitra
 
Oracle database performance tuning
Oracle database performance tuningOracle database performance tuning
Oracle database performance tuningYogiji Creations
 
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified [❤PDF❤] Oracle 19c Database Administration Oracle Simplified
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified ZanderHaney
 
Awr + 12c performance tuning
Awr + 12c performance tuningAwr + 12c performance tuning
Awr + 12c performance tuningAiougVizagChapter
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Vietnam Open Infrastructure User Group
 
Grundlagen Virtualisierung
Grundlagen VirtualisierungGrundlagen Virtualisierung
Grundlagen Virtualisierunginovex GmbH
 
Course 102: Lecture 22: Package Management
Course 102: Lecture 22: Package Management Course 102: Lecture 22: Package Management
Course 102: Lecture 22: Package Management Ahmed El-Arabawy
 
giai-phap-luu-tru-v2
giai-phap-luu-tru-v2giai-phap-luu-tru-v2
giai-phap-luu-tru-v2son2483
 

What's hot (20)

One PDB to go, please!
One PDB to go, please!One PDB to go, please!
One PDB to go, please!
 
Vce vxrail-customer-presentation new
Vce vxrail-customer-presentation newVce vxrail-customer-presentation new
Vce vxrail-customer-presentation new
 
Things you should know about Oracle truncate
Things you should know about Oracle truncateThings you should know about Oracle truncate
Things you should know about Oracle truncate
 
HKG18-318 - OpenAMP Workshop
HKG18-318 - OpenAMP WorkshopHKG18-318 - OpenAMP Workshop
HKG18-318 - OpenAMP Workshop
 
Storage Managment
Storage ManagmentStorage Managment
Storage Managment
 
Oracle RAC - New Generation
Oracle RAC - New GenerationOracle RAC - New Generation
Oracle RAC - New Generation
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
Drilling into Data with Apache Drill
Drilling into Data with Apache DrillDrilling into Data with Apache Drill
Drilling into Data with Apache Drill
 
Deploying PostgreSQL on Kubernetes
Deploying PostgreSQL on KubernetesDeploying PostgreSQL on Kubernetes
Deploying PostgreSQL on Kubernetes
 
Routed Fabrics For Ceph
Routed Fabrics For CephRouted Fabrics For Ceph
Routed Fabrics For Ceph
 
Oracle db performance tuning
Oracle db performance tuningOracle db performance tuning
Oracle db performance tuning
 
Linux: Basics OF Linux
Linux: Basics OF LinuxLinux: Basics OF Linux
Linux: Basics OF Linux
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Oracle database performance tuning
Oracle database performance tuningOracle database performance tuning
Oracle database performance tuning
 
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified [❤PDF❤] Oracle 19c Database Administration Oracle Simplified
[❤PDF❤] Oracle 19c Database Administration Oracle Simplified
 
Awr + 12c performance tuning
Awr + 12c performance tuningAwr + 12c performance tuning
Awr + 12c performance tuning
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
 
Grundlagen Virtualisierung
Grundlagen VirtualisierungGrundlagen Virtualisierung
Grundlagen Virtualisierung
 
Course 102: Lecture 22: Package Management
Course 102: Lecture 22: Package Management Course 102: Lecture 22: Package Management
Course 102: Lecture 22: Package Management
 
giai-phap-luu-tru-v2
giai-phap-luu-tru-v2giai-phap-luu-tru-v2
giai-phap-luu-tru-v2
 

Similar to Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma

Unifying your data management with Hadoop
Unifying your data management with HadoopUnifying your data management with Hadoop
Unifying your data management with HadoopJayant Shekhar
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cRonald Francisco Vargas Quesada
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...Franck Pachot
 
Challenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop EngineChallenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop EngineNicolas Morales
 
Optimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptxOptimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptxJasonTuran2
 
Maximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL AnywhereMaximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL AnywhereSAP Technology
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performanceGuy Harrison
 
Large Data Volume Salesforce experiences
Large Data Volume Salesforce experiencesLarge Data Volume Salesforce experiences
Large Data Volume Salesforce experiencesCidar Mendizabal
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBAKaran Kukreja
 
Zipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering FrameworkZipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering FrameworkDatabricks
 
Percona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuningPercona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuningSergey Petrunya
 
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP benchMultidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP benchRim Moussa
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMorgan Tocker
 
Oracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First TimeOracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First TimeDean Richards
 
Oracle Database Performance Tuning Basics
Oracle Database Performance Tuning BasicsOracle Database Performance Tuning Basics
Oracle Database Performance Tuning Basicsnitin anjankar
 
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Lucidworks
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharmaaioughydchapter
 
Tips tricks to speed nw bi 2009
Tips tricks to speed  nw bi  2009Tips tricks to speed  nw bi  2009
Tips tricks to speed nw bi 2009HawaDia
 

Similar to Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma (20)

Unifying your data management with Hadoop
Unifying your data management with HadoopUnifying your data management with Hadoop
Unifying your data management with Hadoop
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12c
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
 
Challenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop EngineChallenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop Engine
 
Optimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptxOptimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptx
 
Maximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL AnywhereMaximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL Anywhere
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performance
 
Large Data Volume Salesforce experiences
Large Data Volume Salesforce experiencesLarge Data Volume Salesforce experiences
Large Data Volume Salesforce experiences
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
 
Zipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering FrameworkZipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering Framework
 
Percona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuningPercona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuning
 
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP benchMultidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
 
Part5 sql tune
Part5 sql tunePart5 sql tune
Part5 sql tune
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
 
Oracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First TimeOracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First Time
 
Oracle Database Performance Tuning Basics
Oracle Database Performance Tuning BasicsOracle Database Performance Tuning Basics
Oracle Database Performance Tuning Basics
 
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharma
 
Tips tricks to speed nw bi 2009
Tips tricks to speed  nw bi  2009Tips tricks to speed  nw bi  2009
Tips tricks to speed nw bi 2009
 
Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
 

More from Citus Data

Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Citus Data
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Citus Data
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...Citus Data
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Citus Data
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberCitus Data
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncCitus Data
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...Citus Data
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisCitus Data
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Citus Data
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncCitus Data
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Citus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineCitus Data
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberCitus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineCitus Data
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Citus Data
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineCitus Data
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberCitus Data
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoCitus Data
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Citus Data
 

More from Citus Data (20)

Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will Leinweber
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
 

Recently uploaded

Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPTiSEO AI
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfAnubhavMangla3
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireExakis Nelite
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 

Recently uploaded (20)

Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 

Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma

  • 1. Samay Sharma | RailsConf 2019 Optimizing your app by understanding your Postgres database Samay Sharma Solutions Engineering Manager RailsConf 2019 | Minneapolis | April 2019
  • 2. Samay Sharma | RailsConf 20192 • Citus - Open Source Extension to Scale out Postgres • Manage a team of Solutions Engineers • Work with clients • Fun(?) fact : Recently got married J
  • 3. Samay Sharma | RailsConf 2019
  • 4. Samay Sharma | RailsConf 2019 Challenge: Relating issues to the database • Problems you’re facing from an application perspective • Tied to database • How do I find the root cause? • How do I solve it? 4
  • 5. Samay Sharma | RailsConf 2019 Solution: Using Postgres’ statistics • Postgres captures, uses and exposes many statistics. • Monitoring statistics – Statistics about system activity • Query planner statistics – Statistics used by the planner to choose the right query plan. • Server Administrator statistics – Statistics about replication, size of database, tables, etc. • It’s important to know how to use them. 5
  • 6. Samay Sharma | RailsConf 2019 Problem: My application is slow • Symptoms: Application users are seeing slow query performance. • Your monitoring tool shows you that most of the time is going in database calls. • Your page is making 100s of database calls, you don’t know which query is slow. 6
  • 7. Samay Sharma | RailsConf 2019 Cache hit ratio • Gives an idea of how much of your data is coming from the PostgreSQL buffercache. • For transactional apps, want to keep if >95%, ideally around 99%. 7
  • 8. Samay Sharma | RailsConf 2019 How do I measure? SELECT sum(heap_blks_read) as heap_read, sum(heap_blks_hit) as heap_hit, sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio FROM pg_statio_user_tables; 8
  • 9. Samay Sharma | RailsConf 2019 pg_statio_user_tables select * from pg_statio_user_tables where relname = 'tbl'; -[ RECORD 1 ]---+--------- relid | 19049 schemaname | public relname | tbl heap_blks_read | 44260 heap_blks_hit | 10162819 idx_blks_read | 31 idx_blks_hit | 30 toast_blks_read | toast_blks_hit | tidx_blks_read | tidx_blks_hit | 9
  • 10. Samay Sharma | RailsConf 2019 Digging further • If cache hit ratio is low, that could mean a lot of things. • Maybe you have a lot of bloat. • Maybe autovacuum is not tuned well enough. • Maybe your queries are not optimized and you are doing a lot of sequential scans. • Maybe you have a lot of unused indexes. • Or, you actually have less resources and you need to scale out with Citus. 10
  • 11. Samay Sharma | RailsConf 2019 Pg_stat_database select * from pg_stat_database where datname = 'citus'; -[ RECORD 1 ]--+------------------------------ datid | 16385 datname | citus numbackends | 3 xact_commit | 1023786 xact_rollback | 8 blks_read | 45324 blks_hit | 705650570 tup_returned | 675865817 tup_fetched | 367682017 tup_inserted | 10009861 tup_updated | 1738 tup_deleted | 600 conflicts | 0 temp_files | 4 temp_bytes | 340491008 deadlocks | 0 blk_read_time | 59.44 blk_write_time | 0 stats_reset | 2019-01-03 23:07:28.206208+00 11
  • 12. Samay Sharma | RailsConf 2019 Uses for pg_stat_database • Find out rows fetched vs returned by queries to the database • Find out the insert / update / delete ratio for your database • Total number of transactions executed / throughput 12
  • 13. Samay Sharma | RailsConf 2019 pg_stat_user_tables select * from pg_stat_user_tables where relname = 'tbl'; -[ RECORD 1 ]-------+------------------------------ relid | 19049 schemaname | public relname | tbl seq_scan | 3 seq_tup_read | 10000000 idx_scan | 2 idx_tup_fetch | 20000 n_tup_ins | 10000000 n_tup_upd | 0 n_tup_del | 0 n_tup_hot_upd | 0 n_live_tup | 9999977 n_dead_tup | 0 n_mod_since_analyze | 0 last_vacuum | last_autovacuum | last_analyze | last_autoanalyze | 2019-04-28 22:28:42.068611+00 vacuum_count | 0 autovacuum_count | 0 analyze_count | 0 autoanalyze_count | 1 13
  • 14. Samay Sharma | RailsConf 2019 Uses for pg_stat_user_tables • Approximate number of live / dead tuples (rows) in a table • Find out if the table is an insert / update / delete heavy table. • Find out if your table is being auto vacuumed sufficiently or not. • Find out if most scans on the table are sequential scans or index scans. 14
  • 15. Samay Sharma | RailsConf 2019 pg_stat_user_indexes select * from pg_stat_user_indexes where relname = 'tbl'; -[ RECORD 1 ]-+------- relid | 19049 indexrelid | 19052 schemaname | public relname | tbl indexrelname | i1 idx_scan | 4 idx_tup_read | 40000 idx_tup_fetch | 40000 15
  • 16. Samay Sharma | RailsConf 2019 Uses for pg_stat_user_indexes • Figuring out unused indexes • Ratio of index rows read vs fetched 16
  • 17. Samay Sharma | RailsConf 2019 Back to cache hit rate • Understand why you had a bad hit rate • You see database patterns which don’t align with the application – Dive deeper into them • Is there a lot of bloat, are some tables not vacuumed often enough – Tune autovacuum • Are you doing a lot of sequential scans - Create indexes • Maybe you have a lot of unnecessary indexes – Delete them 17
  • 18. Samay Sharma | RailsConf 2019 Generic vs specific optimizations • Follow above practices to make generic optimizations when your database is slow. • How do you speed up a particular webpage? • Or how do you find out what specific queries are slow? 18
  • 19. Samay Sharma | RailsConf 2019 How do I find what queries are slow? 19
  • 20. Samay Sharma | RailsConf 2019 What is `pg_stat_statements` ? • PostgreSQL extension which tracks execution statistics of all SQL statements executed by a server • Normalized so similar queries grouped together • Exposed to user w/ view named `pg_stat_statements` 20
  • 21. Samay Sharma | RailsConf 2019 `pg_stat_statements` view • Query: User_id, db_id, query_id, query text, # of times executed • Timing: min_time, max_time, total_time, mean_time, stddev_time • Shared blocks hit, read, written, dirtied • Local blocks hit, read, written, dirtied • I/O times: total time spent in reading & writing blocks. • Enable track_io_timing to get these, otherwise will be empty 21
  • 22. Samay Sharma | RailsConf 2019 Examples • Top 10 time consuming queries select query, total_time from pg_stat_statements ORDER BY total_time DESC limit 10 • Top 10 queries spending most time on disk reads select query, blk_read_time from pg_stat_statements ORDER BY blk_read_time DESC limit 10 • Top 10 most common queries select query, calls from pg_stat_statements ORDER BY calls DESC limit 10 22
  • 23. Samay Sharma | RailsConf 2019 SELECT rolname, calls, total_time, mean_time, max_time, stddev_time, rows, regexp_replace(query, '[ tn]+', ' ', 'g') AS query_text FROM pg_stat_statements JOIN pg_roles r ON r.oid = userid WHERE calls > 100 AND rolname NOT LIKE '%backup' ORDER BY mean_time DESC LIMIT 15; -[ RECORD 1] ---------------------------------------------------------------------------------------------------- - --------------------------------------------------------------------------------------------------------------- --- rolname | citus calls | 141 total_time | 121404.290523 mean_time | 861.023337042553 max_time | 3718.828292 stddev_time | 913.641891936023 rows | 846 query_text | SELECT ads.campaign_id, COUNT(*) FROM ads JOIN impressions i ON (ads.id = ad_id AND ads.company_id = i.company_id) WHERE ads.campaign_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14) AND ads.company_id = $15 AND seen_at > now()::date GROUP BY ads.campaign_id Source: http://geekmonkey.org/2017/11/optimizing-postgresql-queries-using-pg_stat_statements-and-pg_buffercache/
  • 24. Samay Sharma | RailsConf 2019 EXPLAIN ANALYZE and tune them! EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1; QUERY PLAN ----------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..169247.80 rows=9584 width=8) (actual time=0.641..622.851 rows=10000 loops=1) Filter: (col1 = 1) Rows Removed by Filter: 9990000 Planning time: 0.051 ms Execution time: 623.185 ms (5 rows) 24
  • 25. Samay Sharma | RailsConf 2019 Interesting things about EXPLAIN EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0; QUERY PLAN ---------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..194248.69 rows=100 width=8) (actual time=0.640..630.130 rows=10000 loops=1) Filter: ((col1 = 1) AND (col2 = 0)) Rows Removed by Filter: 9990000 Planning time: 0.072 ms Execution time: 630.467 ms (5 rows) 25
  • 26. Samay Sharma | RailsConf 2019 Estimates vs actual • Postgres tells you what it thinks even in the EXPLAIN plans. • Sometimes reason for slow queries might just be that Postgres has incorrect statistics. Notice and fix those mismatches. • Try to ANALYZE with larger default_statistics_target. 26
  • 27. Samay Sharma | RailsConf 2019 Multi-column statistics • Postgres only captures only single column statistics. • Sometimes, those are not enough because they can’t cover correlations. • But, you can tell PG two columns are related. 27
  • 28. Samay Sharma | RailsConf 2019 CREATE STATISTICS CREATE STATISTICS s1 (dependencies) on col1, col2 from tbl; ANALYZE tbl; EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0; QUERY PLAN ----------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..194247.76 rows=9584 width=8) (actual time=0.638..629.741 rows=10000 loops=1) Filter: ((col1 = 1) AND (col2 = 0)) Rows Removed by Filter: 9990000 Planning time: 0.115 ms Execution time: 630.076 ms (5 rows) 28
  • 29. Samay Sharma | RailsConf 2019 Example of impact 29
  • 30. Samay Sharma | RailsConf 2019 Summary of specific query optimization • Find the right queries to optimize using pg_stat_statements • Use EXPLAIN ANALYZE and tune the slowest parts • Look at differences between estimated and actual and find ways to fix them. 30
  • 31. Samay Sharma | RailsConf 2019 Bonus: Quick Estimates 31
  • 32. Samay Sharma | RailsConf 2019 pg_stats select * from pg_stats where tablename = 'tbl' and attname = 'col1'; -[ RECORD 1 ]----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------ schemaname | public tablename | tbl attname | col1 inherited | f null_frac | 0 avg_width | 4 n_distinct | 1000 most_common_vals | {945,394,677,839,551,636,805,3,743,799,299,967,295,396,400,403,663,740} most_common_freqs | {0.00176667,0.0017,0.0016,0.0016,0.00156667,0.00156667,0.00156667,0.00153333,0.00153333,0.00153333,0.0015,0.0015,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667} histogram_bounds | {0,10,20,29,39,49,58,68,78,88,99,109,119,129,139,149,158,168,177,186,195,205,215,225,235,245,254,263,273,283,292,303,313,323,331,341,350,360,371,380,391,406,415,425,434,443,454,465,474,483 ,492,502,511,521,531,541,550,562,571,581,591,600,610,620,630,642,652,662,672,682,692,702,712,722,731,742,754,763,773,783,792,803,813,824,834,845,855,866,876,886,897,907,917,928,938,948,958 ,969,980,991,999} correlation | 1 most_common_elems | most_common_elem_freqs | elem_count_histogram | 32
  • 33. Samay Sharma | RailsConf 2019 Uses for pg_stats • What is the null ratio for a particular column? • Find the approximate number of distinct values for a column. • What are the most common values for a column and what are the approximate number of rows with those values? 33
  • 34. Samay Sharma | RailsConf 2019 Approximate count • Do you know how to get a very fast approximate count of rows in a table? SELECT relname, reltuples FROM pg_class WHERE relname = 'tbl’; relname | reltuples ---------+----------- tbl | 1e+07 (1 row) 34
  • 35. Samay Sharma | RailsConf 2019 Summary • Postgres exposes a lot of stats to you. In this talk, we covered mainly performance. • Pg_stat_* tables to find index, table, IO, vacuum statistics. • Enable pg_stat_statements to find out your slow queries. 35
  • 36. Samay Sharma | RailsConf 2019 Summary • Use EXPLAIN ANALYZE or EXPLAIN (ANALYZE, BUFFERS) to see what the planner is actually doing. • Use pg_stats and pg_class to get good approximations and understand postgres’ understanding of your data. 36
  • 37. Samay Sharma | RailsConf 2019Samay Sharma | RailsConf 2019 aka.ms/citus-azure.int
  • 38. Samay Sharma | RailsConf 2019 Questions? @citusdata@samay_sharma github.com/citusdata/citus @microsoft
  • 39. Samay Sharma | RailsConf 2019 Useful readings – Based on talk feedback • Measuring Bloat (and more) : https://www.citusdata.com/blog/2017/10/20/monitoring-your-bloat-in-postgres/ • Queries for unused indexes and cache hit ratio : https://www.citusdata.com/blog/2017/09/29/what-performance- can-you-expect-from-postgres/ • Tuning Autovacuum : https://www.citusdata.com/blog/2016/11/04/autovacuum-not-the-enemy/ • Multi-column statistics : https://www.citusdata.com/blog/2018/03/06/postgres-planner-and-its-usage-of- statistics/ 39