SlideShare a Scribd company logo
Samay Sharma | RailsConf 2019
Optimizing your app by understanding
your Postgres database
Samay Sharma
Solutions Engineering Manager
RailsConf 2019 | Minneapolis | April 2019
Samay Sharma | RailsConf 20192
• Citus - Open Source Extension to
Scale out Postgres
• Manage a team of Solutions
Engineers
• Work with clients
• Fun(?) fact : Recently got married J
Samay Sharma | RailsConf 2019
Samay Sharma | RailsConf 2019
Challenge: Relating issues to the database
• Problems you’re facing from an application perspective
• Tied to database
• How do I find the root cause?
• How do I solve it?
4
Samay Sharma | RailsConf 2019
Solution: Using Postgres’ statistics
• Postgres captures, uses and exposes many statistics.
• Monitoring statistics – Statistics about system activity
• Query planner statistics – Statistics used by the planner to choose the right query plan.
• Server Administrator statistics – Statistics about replication, size of database, tables, etc.
• It’s important to know how to use them.
5
Samay Sharma | RailsConf 2019
Problem: My application is slow
• Symptoms: Application users are seeing slow query performance.
• Your monitoring tool shows you that most of the time is going in database
calls.
• Your page is making 100s of database calls, you don’t know which query
is slow.
6
Samay Sharma | RailsConf 2019
Cache hit ratio
• Gives an idea of how much of your data is coming from the PostgreSQL
buffercache.
• For transactional apps, want to keep if >95%, ideally around 99%.
7
Samay Sharma | RailsConf 2019
How do I measure?
SELECT
sum(heap_blks_read) as heap_read,
sum(heap_blks_hit) as heap_hit,
sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio
FROM
pg_statio_user_tables;
8
Samay Sharma | RailsConf 2019
pg_statio_user_tables
select * from pg_statio_user_tables where relname = 'tbl';
-[ RECORD 1 ]---+---------
relid | 19049
schemaname | public
relname | tbl
heap_blks_read | 44260
heap_blks_hit | 10162819
idx_blks_read | 31
idx_blks_hit | 30
toast_blks_read |
toast_blks_hit |
tidx_blks_read |
tidx_blks_hit |
9
Samay Sharma | RailsConf 2019
Digging further
• If cache hit ratio is low, that could mean a lot of things.
• Maybe you have a lot of bloat.
• Maybe autovacuum is not tuned well enough.
• Maybe your queries are not optimized and you are doing a lot of sequential scans.
• Maybe you have a lot of unused indexes.
• Or, you actually have less resources and you need to scale out with Citus.
10
Samay Sharma | RailsConf 2019
Pg_stat_database
select * from pg_stat_database where datname = 'citus';
-[ RECORD 1 ]--+------------------------------
datid | 16385
datname | citus
numbackends | 3
xact_commit | 1023786
xact_rollback | 8
blks_read | 45324
blks_hit | 705650570
tup_returned | 675865817
tup_fetched | 367682017
tup_inserted | 10009861
tup_updated | 1738
tup_deleted | 600
conflicts | 0
temp_files | 4
temp_bytes | 340491008
deadlocks | 0
blk_read_time | 59.44
blk_write_time | 0
stats_reset | 2019-01-03 23:07:28.206208+00
11
Samay Sharma | RailsConf 2019
Uses for pg_stat_database
• Find out rows fetched vs returned by queries to the database
• Find out the insert / update / delete ratio for your database
• Total number of transactions executed / throughput
12
Samay Sharma | RailsConf 2019
pg_stat_user_tables
select * from pg_stat_user_tables where relname = 'tbl';
-[ RECORD 1 ]-------+------------------------------
relid | 19049
schemaname | public
relname | tbl
seq_scan | 3
seq_tup_read | 10000000
idx_scan | 2
idx_tup_fetch | 20000
n_tup_ins | 10000000
n_tup_upd | 0
n_tup_del | 0
n_tup_hot_upd | 0
n_live_tup | 9999977
n_dead_tup | 0
n_mod_since_analyze | 0
last_vacuum |
last_autovacuum |
last_analyze |
last_autoanalyze | 2019-04-28 22:28:42.068611+00
vacuum_count | 0
autovacuum_count | 0
analyze_count | 0
autoanalyze_count | 1
13
Samay Sharma | RailsConf 2019
Uses for pg_stat_user_tables
• Approximate number of live / dead tuples (rows) in a table
• Find out if the table is an insert / update / delete heavy table.
• Find out if your table is being auto vacuumed sufficiently or not.
• Find out if most scans on the table are sequential scans or index scans.
14
Samay Sharma | RailsConf 2019
pg_stat_user_indexes
select * from pg_stat_user_indexes where relname = 'tbl';
-[ RECORD 1 ]-+-------
relid | 19049
indexrelid | 19052
schemaname | public
relname | tbl
indexrelname | i1
idx_scan | 4
idx_tup_read | 40000
idx_tup_fetch | 40000
15
Samay Sharma | RailsConf 2019
Uses for pg_stat_user_indexes
• Figuring out unused indexes
• Ratio of index rows read vs fetched
16
Samay Sharma | RailsConf 2019
Back to cache hit rate
• Understand why you had a bad hit rate
• You see database patterns which don’t align with the application – Dive deeper into them
• Is there a lot of bloat, are some tables not vacuumed often enough – Tune autovacuum
• Are you doing a lot of sequential scans - Create indexes
• Maybe you have a lot of unnecessary indexes – Delete them
17
Samay Sharma | RailsConf 2019
Generic vs specific optimizations
• Follow above practices to make generic optimizations when your
database is slow.
• How do you speed up a particular webpage?
• Or how do you find out what specific queries are slow?
18
Samay Sharma | RailsConf 2019
How do I find what queries are slow?
19
Samay Sharma | RailsConf 2019
What is `pg_stat_statements` ?
• PostgreSQL extension which tracks execution statistics of all SQL
statements executed by a server
• Normalized so similar queries grouped together
• Exposed to user w/ view named `pg_stat_statements`
20
Samay Sharma | RailsConf 2019
`pg_stat_statements` view
• Query: User_id, db_id, query_id, query text, # of times executed
• Timing: min_time, max_time, total_time, mean_time, stddev_time
• Shared blocks hit, read, written, dirtied
• Local blocks hit, read, written, dirtied
• I/O times: total time spent in reading & writing blocks.
• Enable track_io_timing to get these, otherwise will be empty
21
Samay Sharma | RailsConf 2019
Examples
• Top 10 time consuming queries
select query, total_time from pg_stat_statements ORDER BY total_time DESC limit 10
• Top 10 queries spending most time on disk reads
select query, blk_read_time from pg_stat_statements ORDER BY blk_read_time DESC limit 10
• Top 10 most common queries
select query, calls from pg_stat_statements ORDER BY calls DESC limit 10
22
Samay Sharma | RailsConf 2019
SELECT rolname,
calls,
total_time,
mean_time,
max_time,
stddev_time,
rows,
regexp_replace(query, '[ tn]+', ' ', 'g') AS query_text
FROM pg_stat_statements
JOIN pg_roles r ON r.oid = userid
WHERE calls > 100
AND rolname NOT LIKE '%backup'
ORDER BY mean_time DESC
LIMIT 15;
-[ RECORD 1] ----------------------------------------------------------------------------------------------------
- ---------------------------------------------------------------------------------------------------------------
---
rolname | citus
calls | 141
total_time | 121404.290523
mean_time | 861.023337042553
max_time | 3718.828292
stddev_time | 913.641891936023
rows | 846
query_text | SELECT ads.campaign_id, COUNT(*) FROM ads JOIN impressions i ON (ads.id = ad_id AND ads.company_id
= i.company_id) WHERE ads.campaign_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14) AND ads.company_id =
$15 AND seen_at > now()::date GROUP BY ads.campaign_id
Source: http://geekmonkey.org/2017/11/optimizing-postgresql-queries-using-pg_stat_statements-and-pg_buffercache/
Samay Sharma | RailsConf 2019
EXPLAIN ANALYZE and tune them!
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..169247.80 rows=9584 width=8) (actual time=0.641..622.851 rows=10000 loops=1)
Filter: (col1 = 1)
Rows Removed by Filter: 9990000
Planning time: 0.051 ms
Execution time: 623.185 ms
(5 rows)
24
Samay Sharma | RailsConf 2019
Interesting things about EXPLAIN
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0;
QUERY PLAN
----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..194248.69 rows=100 width=8) (actual time=0.640..630.130 rows=10000 loops=1)
Filter: ((col1 = 1) AND (col2 = 0))
Rows Removed by Filter: 9990000
Planning time: 0.072 ms
Execution time: 630.467 ms
(5 rows)
25
Samay Sharma | RailsConf 2019
Estimates vs actual
• Postgres tells you what it thinks even in the EXPLAIN plans.
• Sometimes reason for slow queries might just be that Postgres has
incorrect statistics. Notice and fix those mismatches.
• Try to ANALYZE with larger default_statistics_target.
26
Samay Sharma | RailsConf 2019
Multi-column statistics
• Postgres only captures only single column statistics.
• Sometimes, those are not enough because they can’t cover correlations.
• But, you can tell PG two columns are related.
27
Samay Sharma | RailsConf 2019
CREATE STATISTICS
CREATE STATISTICS s1 (dependencies) on col1, col2 from tbl;
ANALYZE tbl;
EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------
Seq Scan on tbl (cost=0.00..194247.76 rows=9584 width=8) (actual time=0.638..629.741 rows=10000 loops=1)
Filter: ((col1 = 1) AND (col2 = 0))
Rows Removed by Filter: 9990000
Planning time: 0.115 ms
Execution time: 630.076 ms
(5 rows)
28
Samay Sharma | RailsConf 2019
Example of impact
29
Samay Sharma | RailsConf 2019
Summary of specific query optimization
• Find the right queries to optimize using pg_stat_statements
• Use EXPLAIN ANALYZE and tune the slowest parts
• Look at differences between estimated and actual and find ways to fix
them.
30
Samay Sharma | RailsConf 2019
Bonus: Quick Estimates
31
Samay Sharma | RailsConf 2019
pg_stats
select * from pg_stats where tablename = 'tbl' and attname = 'col1';
-[ RECORD 1 ]----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------
schemaname | public
tablename | tbl
attname | col1
inherited | f
null_frac | 0
avg_width | 4
n_distinct | 1000
most_common_vals | {945,394,677,839,551,636,805,3,743,799,299,967,295,396,400,403,663,740}
most_common_freqs |
{0.00176667,0.0017,0.0016,0.0016,0.00156667,0.00156667,0.00156667,0.00153333,0.00153333,0.00153333,0.0015,0.0015,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667}
histogram_bounds |
{0,10,20,29,39,49,58,68,78,88,99,109,119,129,139,149,158,168,177,186,195,205,215,225,235,245,254,263,273,283,292,303,313,323,331,341,350,360,371,380,391,406,415,425,434,443,454,465,474,483
,492,502,511,521,531,541,550,562,571,581,591,600,610,620,630,642,652,662,672,682,692,702,712,722,731,742,754,763,773,783,792,803,813,824,834,845,855,866,876,886,897,907,917,928,938,948,958
,969,980,991,999}
correlation | 1
most_common_elems |
most_common_elem_freqs |
elem_count_histogram |
32
Samay Sharma | RailsConf 2019
Uses for pg_stats
• What is the null ratio for a particular column?
• Find the approximate number of distinct values for a column.
• What are the most common values for a column and what are the
approximate number of rows with those values?
33
Samay Sharma | RailsConf 2019
Approximate count
• Do you know how to get a very fast approximate count of rows in a table?
SELECT
relname, reltuples
FROM
pg_class
WHERE
relname = 'tbl’;
relname | reltuples
---------+-----------
tbl | 1e+07
(1 row)
34
Samay Sharma | RailsConf 2019
Summary
• Postgres exposes a lot of stats to you. In this talk, we covered mainly
performance.
• Pg_stat_* tables to find index, table, IO, vacuum statistics.
• Enable pg_stat_statements to find out your slow queries.
35
Samay Sharma | RailsConf 2019
Summary
• Use EXPLAIN ANALYZE or EXPLAIN (ANALYZE, BUFFERS) to see what
the planner is actually doing.
• Use pg_stats and pg_class to get good approximations and understand
postgres’ understanding of your data.
36
Samay Sharma | RailsConf 2019Samay Sharma | RailsConf 2019
aka.ms/citus-azure.int
Samay Sharma | RailsConf 2019
Questions?
@citusdata@samay_sharma
github.com/citusdata/citus
@microsoft
Samay Sharma | RailsConf 2019
Useful readings – Based on talk feedback
• Measuring Bloat (and more) : https://www.citusdata.com/blog/2017/10/20/monitoring-your-bloat-in-postgres/
• Queries for unused indexes and cache hit ratio : https://www.citusdata.com/blog/2017/09/29/what-performance-
can-you-expect-from-postgres/
• Tuning Autovacuum : https://www.citusdata.com/blog/2016/11/04/autovacuum-not-the-enemy/
• Multi-column statistics : https://www.citusdata.com/blog/2018/03/06/postgres-planner-and-its-usage-of-
statistics/
39

More Related Content

What's hot

How to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelinHow to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelin
Tiago Simões
 
MySQL Cluster - visão geral
MySQL Cluster - visão geralMySQL Cluster - visão geral
MySQL Cluster - visão geral
MySQL Brasil
 
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark ApplicationsTop 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
Spark Summit
 
Reliable Windows Heap Exploits
Reliable Windows Heap ExploitsReliable Windows Heap Exploits
Reliable Windows Heap Exploits
amiable_indian
 
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
ScyllaDB
 
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
DataStax
 
Kernel Recipes 2019 - Faster IO through io_uring
Kernel Recipes 2019 - Faster IO through io_uringKernel Recipes 2019 - Faster IO through io_uring
Kernel Recipes 2019 - Faster IO through io_uring
Anne Nicolas
 
How to be Successful with Scylla
How to be Successful with ScyllaHow to be Successful with Scylla
How to be Successful with Scylla
ScyllaDB
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
MIJIN AN
 
Presto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 UpdatesPresto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 Updates
Taro L. Saito
 
Linux Kernel I/O Schedulers
Linux Kernel I/O SchedulersLinux Kernel I/O Schedulers
Linux Kernel I/O Schedulers
RajKumar Rampelli
 
How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)
DataStax Academy
 
Node Labels in YARN
Node Labels in YARNNode Labels in YARN
Node Labels in YARN
DataWorks Summit
 
Ceph and RocksDB
Ceph and RocksDBCeph and RocksDB
Ceph and RocksDB
Sage Weil
 
Meet Apache HBase - 2.0
Meet Apache HBase - 2.0Meet Apache HBase - 2.0
Meet Apache HBase - 2.0
DataWorks Summit
 
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
Chris Bolman
 
Cephfs jewel mds performance benchmark
Cephfs jewel mds performance benchmarkCephfs jewel mds performance benchmark
Cephfs jewel mds performance benchmark
Xiaoxi Chen
 
SQL Server Clustering and High Availability
SQL Server Clustering and High AvailabilitySQL Server Clustering and High Availability
SQL Server Clustering and High Availability
► Supreme Mandal ◄
 
Curso Adabas
Curso AdabasCurso Adabas
Curso Adabas
neigao
 
Cassandra data structures and algorithms
Cassandra data structures and algorithmsCassandra data structures and algorithms
Cassandra data structures and algorithms
Duyhai Doan
 

What's hot (20)

How to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelinHow to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelin
 
MySQL Cluster - visão geral
MySQL Cluster - visão geralMySQL Cluster - visão geral
MySQL Cluster - visão geral
 
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark ApplicationsTop 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
 
Reliable Windows Heap Exploits
Reliable Windows Heap ExploitsReliable Windows Heap Exploits
Reliable Windows Heap Exploits
 
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
 
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
The Missing Manual for Leveled Compaction Strategy (Wei Deng & Ryan Svihla, D...
 
Kernel Recipes 2019 - Faster IO through io_uring
Kernel Recipes 2019 - Faster IO through io_uringKernel Recipes 2019 - Faster IO through io_uring
Kernel Recipes 2019 - Faster IO through io_uring
 
How to be Successful with Scylla
How to be Successful with ScyllaHow to be Successful with Scylla
How to be Successful with Scylla
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
 
Presto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 UpdatesPresto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 Updates
 
Linux Kernel I/O Schedulers
Linux Kernel I/O SchedulersLinux Kernel I/O Schedulers
Linux Kernel I/O Schedulers
 
How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)
 
Node Labels in YARN
Node Labels in YARNNode Labels in YARN
Node Labels in YARN
 
Ceph and RocksDB
Ceph and RocksDBCeph and RocksDB
Ceph and RocksDB
 
Meet Apache HBase - 2.0
Meet Apache HBase - 2.0Meet Apache HBase - 2.0
Meet Apache HBase - 2.0
 
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)
 
Cephfs jewel mds performance benchmark
Cephfs jewel mds performance benchmarkCephfs jewel mds performance benchmark
Cephfs jewel mds performance benchmark
 
SQL Server Clustering and High Availability
SQL Server Clustering and High AvailabilitySQL Server Clustering and High Availability
SQL Server Clustering and High Availability
 
Curso Adabas
Curso AdabasCurso Adabas
Curso Adabas
 
Cassandra data structures and algorithms
Cassandra data structures and algorithmsCassandra data structures and algorithms
Cassandra data structures and algorithms
 

Similar to Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma

Unifying your data management with Hadoop
Unifying your data management with HadoopUnifying your data management with Hadoop
Unifying your data management with Hadoop
Jayant Shekhar
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Ronald Francisco Vargas Quesada
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
Franck Pachot
 
Challenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop EngineChallenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop Engine
Nicolas Morales
 
Optimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptxOptimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptx
JasonTuran2
 
Maximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL AnywhereMaximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL Anywhere
SAP Technology
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performance
Guy Harrison
 
Large Data Volume Salesforce experiences
Large Data Volume Salesforce experiencesLarge Data Volume Salesforce experiences
Large Data Volume Salesforce experiences
Cidar Mendizabal
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
Karan Kukreja
 
Zipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering FrameworkZipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering Framework
Databricks
 
Percona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuningPercona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuning
Sergey Petrunya
 
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP benchMultidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
Rim Moussa
 
Part5 sql tune
Part5 sql tunePart5 sql tune
Part5 sql tune
Maria Colgan
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
Morgan Tocker
 
Oracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First TimeOracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First Time
Dean Richards
 
Oracle Database Performance Tuning Basics
Oracle Database Performance Tuning BasicsOracle Database Performance Tuning Basics
Oracle Database Performance Tuning Basics
nitin anjankar
 
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Lucidworks
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharma
aioughydchapter
 
Tips tricks to speed nw bi 2009
Tips tricks to speed  nw bi  2009Tips tricks to speed  nw bi  2009
Tips tricks to speed nw bi 2009
HawaDia
 
Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
Ligaya Turmelle
 

Similar to Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma (20)

Unifying your data management with Hadoop
Unifying your data management with HadoopUnifying your data management with Hadoop
Unifying your data management with Hadoop
 
Presentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12cPresentación Oracle Database Migración consideraciones 10g/11g/12c
Presentación Oracle Database Migración consideraciones 10g/11g/12c
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
 
Challenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop EngineChallenges of Building a First Class SQL-on-Hadoop Engine
Challenges of Building a First Class SQL-on-Hadoop Engine
 
Optimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptxOptimizing Application Performance - 2022.pptx
Optimizing Application Performance - 2022.pptx
 
Maximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL AnywhereMaximizing Database Tuning in SAP SQL Anywhere
Maximizing Database Tuning in SAP SQL Anywhere
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performance
 
Large Data Volume Salesforce experiences
Large Data Volume Salesforce experiencesLarge Data Volume Salesforce experiences
Large Data Volume Salesforce experiences
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
 
Zipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering FrameworkZipline - A Declarative Feature Engineering Framework
Zipline - A Declarative Feature Engineering Framework
 
Percona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuningPercona live-2012-optimizer-tuning
Percona live-2012-optimizer-tuning
 
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP benchMultidimensional DB design, revolving TPC-H benchmark into OLAP bench
Multidimensional DB design, revolving TPC-H benchmark into OLAP bench
 
Part5 sql tune
Part5 sql tunePart5 sql tune
Part5 sql tune
 
MySQL Performance Metrics that Matter
MySQL Performance Metrics that MatterMySQL Performance Metrics that Matter
MySQL Performance Metrics that Matter
 
Oracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First TimeOracle Query Tuning Tips - Get it Right the First Time
Oracle Query Tuning Tips - Get it Right the First Time
 
Oracle Database Performance Tuning Basics
Oracle Database Performance Tuning BasicsOracle Database Performance Tuning Basics
Oracle Database Performance Tuning Basics
 
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharma
 
Tips tricks to speed nw bi 2009
Tips tricks to speed  nw bi  2009Tips tricks to speed  nw bi  2009
Tips tricks to speed nw bi 2009
 
Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
 

More from Citus Data

Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Citus Data
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Citus Data
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
Citus Data
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Citus Data
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Citus Data
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will Leinweber
Citus Data
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Citus Data
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
Citus Data
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Citus Data
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Citus Data
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
Citus Data
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Citus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
Citus Data
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
Citus Data
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
Citus Data
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Citus Data
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
Citus Data
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
Citus Data
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Citus Data
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Citus Data
 

More from Citus Data (20)

Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
 
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...
 
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensWhats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
 
When it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will LeinweberWhen it all goes wrong | PGConf EU 2019 | Will Leinweber
When it all goes wrong | PGConf EU 2019 | Will Leinweber
 
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncAmazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc
 
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...
 
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisDeep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis
 
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...
 
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncA story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc
 
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...
 
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine
 
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberWhen it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber
 
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri FontaineThe Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
 
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...
 
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineHow to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine
 
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberWhen it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber
 
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoWhy PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano
 
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...
 

Recently uploaded

Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 

Recently uploaded (20)

Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 

Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay Sharma

  • 1. Samay Sharma | RailsConf 2019 Optimizing your app by understanding your Postgres database Samay Sharma Solutions Engineering Manager RailsConf 2019 | Minneapolis | April 2019
  • 2. Samay Sharma | RailsConf 20192 • Citus - Open Source Extension to Scale out Postgres • Manage a team of Solutions Engineers • Work with clients • Fun(?) fact : Recently got married J
  • 3. Samay Sharma | RailsConf 2019
  • 4. Samay Sharma | RailsConf 2019 Challenge: Relating issues to the database • Problems you’re facing from an application perspective • Tied to database • How do I find the root cause? • How do I solve it? 4
  • 5. Samay Sharma | RailsConf 2019 Solution: Using Postgres’ statistics • Postgres captures, uses and exposes many statistics. • Monitoring statistics – Statistics about system activity • Query planner statistics – Statistics used by the planner to choose the right query plan. • Server Administrator statistics – Statistics about replication, size of database, tables, etc. • It’s important to know how to use them. 5
  • 6. Samay Sharma | RailsConf 2019 Problem: My application is slow • Symptoms: Application users are seeing slow query performance. • Your monitoring tool shows you that most of the time is going in database calls. • Your page is making 100s of database calls, you don’t know which query is slow. 6
  • 7. Samay Sharma | RailsConf 2019 Cache hit ratio • Gives an idea of how much of your data is coming from the PostgreSQL buffercache. • For transactional apps, want to keep if >95%, ideally around 99%. 7
  • 8. Samay Sharma | RailsConf 2019 How do I measure? SELECT sum(heap_blks_read) as heap_read, sum(heap_blks_hit) as heap_hit, sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio FROM pg_statio_user_tables; 8
  • 9. Samay Sharma | RailsConf 2019 pg_statio_user_tables select * from pg_statio_user_tables where relname = 'tbl'; -[ RECORD 1 ]---+--------- relid | 19049 schemaname | public relname | tbl heap_blks_read | 44260 heap_blks_hit | 10162819 idx_blks_read | 31 idx_blks_hit | 30 toast_blks_read | toast_blks_hit | tidx_blks_read | tidx_blks_hit | 9
  • 10. Samay Sharma | RailsConf 2019 Digging further • If cache hit ratio is low, that could mean a lot of things. • Maybe you have a lot of bloat. • Maybe autovacuum is not tuned well enough. • Maybe your queries are not optimized and you are doing a lot of sequential scans. • Maybe you have a lot of unused indexes. • Or, you actually have less resources and you need to scale out with Citus. 10
  • 11. Samay Sharma | RailsConf 2019 Pg_stat_database select * from pg_stat_database where datname = 'citus'; -[ RECORD 1 ]--+------------------------------ datid | 16385 datname | citus numbackends | 3 xact_commit | 1023786 xact_rollback | 8 blks_read | 45324 blks_hit | 705650570 tup_returned | 675865817 tup_fetched | 367682017 tup_inserted | 10009861 tup_updated | 1738 tup_deleted | 600 conflicts | 0 temp_files | 4 temp_bytes | 340491008 deadlocks | 0 blk_read_time | 59.44 blk_write_time | 0 stats_reset | 2019-01-03 23:07:28.206208+00 11
  • 12. Samay Sharma | RailsConf 2019 Uses for pg_stat_database • Find out rows fetched vs returned by queries to the database • Find out the insert / update / delete ratio for your database • Total number of transactions executed / throughput 12
  • 13. Samay Sharma | RailsConf 2019 pg_stat_user_tables select * from pg_stat_user_tables where relname = 'tbl'; -[ RECORD 1 ]-------+------------------------------ relid | 19049 schemaname | public relname | tbl seq_scan | 3 seq_tup_read | 10000000 idx_scan | 2 idx_tup_fetch | 20000 n_tup_ins | 10000000 n_tup_upd | 0 n_tup_del | 0 n_tup_hot_upd | 0 n_live_tup | 9999977 n_dead_tup | 0 n_mod_since_analyze | 0 last_vacuum | last_autovacuum | last_analyze | last_autoanalyze | 2019-04-28 22:28:42.068611+00 vacuum_count | 0 autovacuum_count | 0 analyze_count | 0 autoanalyze_count | 1 13
  • 14. Samay Sharma | RailsConf 2019 Uses for pg_stat_user_tables • Approximate number of live / dead tuples (rows) in a table • Find out if the table is an insert / update / delete heavy table. • Find out if your table is being auto vacuumed sufficiently or not. • Find out if most scans on the table are sequential scans or index scans. 14
  • 15. Samay Sharma | RailsConf 2019 pg_stat_user_indexes select * from pg_stat_user_indexes where relname = 'tbl'; -[ RECORD 1 ]-+------- relid | 19049 indexrelid | 19052 schemaname | public relname | tbl indexrelname | i1 idx_scan | 4 idx_tup_read | 40000 idx_tup_fetch | 40000 15
  • 16. Samay Sharma | RailsConf 2019 Uses for pg_stat_user_indexes • Figuring out unused indexes • Ratio of index rows read vs fetched 16
  • 17. Samay Sharma | RailsConf 2019 Back to cache hit rate • Understand why you had a bad hit rate • You see database patterns which don’t align with the application – Dive deeper into them • Is there a lot of bloat, are some tables not vacuumed often enough – Tune autovacuum • Are you doing a lot of sequential scans - Create indexes • Maybe you have a lot of unnecessary indexes – Delete them 17
  • 18. Samay Sharma | RailsConf 2019 Generic vs specific optimizations • Follow above practices to make generic optimizations when your database is slow. • How do you speed up a particular webpage? • Or how do you find out what specific queries are slow? 18
  • 19. Samay Sharma | RailsConf 2019 How do I find what queries are slow? 19
  • 20. Samay Sharma | RailsConf 2019 What is `pg_stat_statements` ? • PostgreSQL extension which tracks execution statistics of all SQL statements executed by a server • Normalized so similar queries grouped together • Exposed to user w/ view named `pg_stat_statements` 20
  • 21. Samay Sharma | RailsConf 2019 `pg_stat_statements` view • Query: User_id, db_id, query_id, query text, # of times executed • Timing: min_time, max_time, total_time, mean_time, stddev_time • Shared blocks hit, read, written, dirtied • Local blocks hit, read, written, dirtied • I/O times: total time spent in reading & writing blocks. • Enable track_io_timing to get these, otherwise will be empty 21
  • 22. Samay Sharma | RailsConf 2019 Examples • Top 10 time consuming queries select query, total_time from pg_stat_statements ORDER BY total_time DESC limit 10 • Top 10 queries spending most time on disk reads select query, blk_read_time from pg_stat_statements ORDER BY blk_read_time DESC limit 10 • Top 10 most common queries select query, calls from pg_stat_statements ORDER BY calls DESC limit 10 22
  • 23. Samay Sharma | RailsConf 2019 SELECT rolname, calls, total_time, mean_time, max_time, stddev_time, rows, regexp_replace(query, '[ tn]+', ' ', 'g') AS query_text FROM pg_stat_statements JOIN pg_roles r ON r.oid = userid WHERE calls > 100 AND rolname NOT LIKE '%backup' ORDER BY mean_time DESC LIMIT 15; -[ RECORD 1] ---------------------------------------------------------------------------------------------------- - --------------------------------------------------------------------------------------------------------------- --- rolname | citus calls | 141 total_time | 121404.290523 mean_time | 861.023337042553 max_time | 3718.828292 stddev_time | 913.641891936023 rows | 846 query_text | SELECT ads.campaign_id, COUNT(*) FROM ads JOIN impressions i ON (ads.id = ad_id AND ads.company_id = i.company_id) WHERE ads.campaign_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14) AND ads.company_id = $15 AND seen_at > now()::date GROUP BY ads.campaign_id Source: http://geekmonkey.org/2017/11/optimizing-postgresql-queries-using-pg_stat_statements-and-pg_buffercache/
  • 24. Samay Sharma | RailsConf 2019 EXPLAIN ANALYZE and tune them! EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1; QUERY PLAN ----------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..169247.80 rows=9584 width=8) (actual time=0.641..622.851 rows=10000 loops=1) Filter: (col1 = 1) Rows Removed by Filter: 9990000 Planning time: 0.051 ms Execution time: 623.185 ms (5 rows) 24
  • 25. Samay Sharma | RailsConf 2019 Interesting things about EXPLAIN EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0; QUERY PLAN ---------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..194248.69 rows=100 width=8) (actual time=0.640..630.130 rows=10000 loops=1) Filter: ((col1 = 1) AND (col2 = 0)) Rows Removed by Filter: 9990000 Planning time: 0.072 ms Execution time: 630.467 ms (5 rows) 25
  • 26. Samay Sharma | RailsConf 2019 Estimates vs actual • Postgres tells you what it thinks even in the EXPLAIN plans. • Sometimes reason for slow queries might just be that Postgres has incorrect statistics. Notice and fix those mismatches. • Try to ANALYZE with larger default_statistics_target. 26
  • 27. Samay Sharma | RailsConf 2019 Multi-column statistics • Postgres only captures only single column statistics. • Sometimes, those are not enough because they can’t cover correlations. • But, you can tell PG two columns are related. 27
  • 28. Samay Sharma | RailsConf 2019 CREATE STATISTICS CREATE STATISTICS s1 (dependencies) on col1, col2 from tbl; ANALYZE tbl; EXPLAIN ANALYZE SELECT * FROM tbl where col1 = 1 and col2 = 0; QUERY PLAN ----------------------------------------------------------------------------------------------------------- Seq Scan on tbl (cost=0.00..194247.76 rows=9584 width=8) (actual time=0.638..629.741 rows=10000 loops=1) Filter: ((col1 = 1) AND (col2 = 0)) Rows Removed by Filter: 9990000 Planning time: 0.115 ms Execution time: 630.076 ms (5 rows) 28
  • 29. Samay Sharma | RailsConf 2019 Example of impact 29
  • 30. Samay Sharma | RailsConf 2019 Summary of specific query optimization • Find the right queries to optimize using pg_stat_statements • Use EXPLAIN ANALYZE and tune the slowest parts • Look at differences between estimated and actual and find ways to fix them. 30
  • 31. Samay Sharma | RailsConf 2019 Bonus: Quick Estimates 31
  • 32. Samay Sharma | RailsConf 2019 pg_stats select * from pg_stats where tablename = 'tbl' and attname = 'col1'; -[ RECORD 1 ]----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------ schemaname | public tablename | tbl attname | col1 inherited | f null_frac | 0 avg_width | 4 n_distinct | 1000 most_common_vals | {945,394,677,839,551,636,805,3,743,799,299,967,295,396,400,403,663,740} most_common_freqs | {0.00176667,0.0017,0.0016,0.0016,0.00156667,0.00156667,0.00156667,0.00153333,0.00153333,0.00153333,0.0015,0.0015,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667,0.00146667} histogram_bounds | {0,10,20,29,39,49,58,68,78,88,99,109,119,129,139,149,158,168,177,186,195,205,215,225,235,245,254,263,273,283,292,303,313,323,331,341,350,360,371,380,391,406,415,425,434,443,454,465,474,483 ,492,502,511,521,531,541,550,562,571,581,591,600,610,620,630,642,652,662,672,682,692,702,712,722,731,742,754,763,773,783,792,803,813,824,834,845,855,866,876,886,897,907,917,928,938,948,958 ,969,980,991,999} correlation | 1 most_common_elems | most_common_elem_freqs | elem_count_histogram | 32
  • 33. Samay Sharma | RailsConf 2019 Uses for pg_stats • What is the null ratio for a particular column? • Find the approximate number of distinct values for a column. • What are the most common values for a column and what are the approximate number of rows with those values? 33
  • 34. Samay Sharma | RailsConf 2019 Approximate count • Do you know how to get a very fast approximate count of rows in a table? SELECT relname, reltuples FROM pg_class WHERE relname = 'tbl’; relname | reltuples ---------+----------- tbl | 1e+07 (1 row) 34
  • 35. Samay Sharma | RailsConf 2019 Summary • Postgres exposes a lot of stats to you. In this talk, we covered mainly performance. • Pg_stat_* tables to find index, table, IO, vacuum statistics. • Enable pg_stat_statements to find out your slow queries. 35
  • 36. Samay Sharma | RailsConf 2019 Summary • Use EXPLAIN ANALYZE or EXPLAIN (ANALYZE, BUFFERS) to see what the planner is actually doing. • Use pg_stats and pg_class to get good approximations and understand postgres’ understanding of your data. 36
  • 37. Samay Sharma | RailsConf 2019Samay Sharma | RailsConf 2019 aka.ms/citus-azure.int
  • 38. Samay Sharma | RailsConf 2019 Questions? @citusdata@samay_sharma github.com/citusdata/citus @microsoft
  • 39. Samay Sharma | RailsConf 2019 Useful readings – Based on talk feedback • Measuring Bloat (and more) : https://www.citusdata.com/blog/2017/10/20/monitoring-your-bloat-in-postgres/ • Queries for unused indexes and cache hit ratio : https://www.citusdata.com/blog/2017/09/29/what-performance- can-you-expect-from-postgres/ • Tuning Autovacuum : https://www.citusdata.com/blog/2016/11/04/autovacuum-not-the-enemy/ • Multi-column statistics : https://www.citusdata.com/blog/2018/03/06/postgres-planner-and-its-usage-of- statistics/ 39