SlideShare a Scribd company logo
PostgreSQL worst practices
at FOSDEM PGDay Brussels 2017
Ilya Kosmodemiansky
ik@postgresql-consulting.com
Best practices are just boring
• Never follow them, try worst practices
• Only those practices can really help you to screw the things up
most effectively
• PostgreSQL consultants are nice people, so try to make them
happy
How it works?
• I have a list, a little bit more than 100 worst practices
• I do not make this stuff up, all of them are real-life examples
• I reshuffle my list every time before presenting and extract
some amount of examples
• Well, there are some things, which I like more or less, so it is
not a very honest shuffle
0. Do not use indexes (a test one!)
• Basically, there is no difference between full table scan and
index scan
• You can check that. Just insert 10 rows into a test table on
your test server and compare.
• Nobody deals with more than 10 row tables in production!
1. Use ORM
• All databases share the same syntax
• You must write database-independent code
• Are there any benefits, which are based on database specific
features?
• It always good to learn a new complicated technology
2. Move joins to your application
• Just select * a couple of tables into the application written in
your favorite programming language
• Than join them at the application level
2. Move joins to your application
• Just select * a couple of tables into the application written in
your favorite programming language
• Than join them at the application level
• Now you only need to implement nested loop join, hash join
and merge join as well as query optimizer and page cache
3. Be in trend, be schema-less
• You do not need to design the schema
• You need only one table, two columns: id bigserial and extra
jsonb
• JSONB datatype is pretty effective in PostgreSQL, you can
search in it just like in a well-structured table
• Even if you put a 100M of JSON in it
• Even if you have 1000+ tps
4. Be agile, use EAV
• You need only 3 tables: entity, attribute, value
4. Be agile, use EAV
• You need only 3 tables: entity, attribute, value
• At some point add the 4th: attribute_type
4. Be agile, use EAV
• You need only 3 tables: entity, attribute, value
• At some point add the 4th: attribute_type
• Whet it starts to work slow, just call those four tables The
Core and add 1000+ tables with denormalized data
4. Be agile, use EAV
• You need only 3 tables: entity, attribute, value
• At some point add the 4th: attribute_type
• Whet it starts to work slow, just call those four tables The
Core and add 1000+ tables with denormalized data
• If it is not enough, you can always add value_version
5. Try to create as many indexes as you can
• Indexes consume no disk space
• Indexes consume no shared_bufers
• There is no overhead on DML if one and every column in a
table covered with bunch of indexes
• Optimizer will definitely choose your index once you created it
• Keep calm and create more indexes
6. Always keep all your time series data
• Time series data like tables with logs or session history should
be never deleted, aggregated or archived, you always need to
keep it all
6. Always keep all your time series data
• Time series data like tables with logs or session history should
be never deleted, aggregated or archived, you always need to
keep it all
• You will always know where to check, if you run out of disk
space
6. Always keep all your time series data
• Time series data like tables with logs or session history should
be never deleted, aggregated or archived, you always need to
keep it all
• You will always know where to check, if you run out of disk
space
• You can always call that Big Data
6. Always keep all your time series data
• Time series data like tables with logs or session history should
be never deleted, aggregated or archived, you always need to
keep it all
• You will always know where to check, if you run out of disk
space
• You can always call that Big Data
• Solve the problem using partitioning... one partition for an
hour or for a minute
7. Turn autovacuum off
• It is quite auxiliary process, you can easily stop it
• There is no problem at all to have 100Gb data in a database
which is 1Tb in size
• 2-3Tb RAM servers are cheap, IO is a fastest thing in modern
computing
• Besides of that, everyone likes BigData
8. Keep master and slave on different hardware
• That will maximize the possibility of unsuccessful failover
8. Keep master and slave on different hardware
• That will maximize the possibility of unsuccessful failover
• To make things worser, you can change only slave-related
parameters at slave, leaving defaults for shared_buffers etc.
9. Put a synchronous replica to remote DC
• Indeed! That will maximize availability!
9. Put a synchronous replica to remote DC
• Indeed! That will maximize availability!
• Especially, if you put the replica to another continent
10. Reinvent Slony
• If you need some data replication to another database, try to
implement it from scratch
10. Reinvent Slony
• If you need some data replication to another database, try to
implement it from scratch
• That allows you to run into all problems, PostgreSQL have
had since introducing Slony
11. Use as many count(*) as you can
• Figure 301083021830123921 is very informative for the end
user
• If it changes in a second to 30108302894839434020, it is still
informative
• select count(*) from sometable is a quite light-weighted query
• Tuple estimation from pg_catalog can never be precise
enough for you
12. Never use graphical monitoring
• You do not need graphs
• Because it is an easy task to guess what was happened
yesterday at 2 a.m. using command line and grep only
13. Never use Foreign Keys
(Use local produced instead!)
• Consistency control at application level always works as
expected
• You will never get data inconsistency without constraints
• Even if you already have a bullet proof framework to maintain
consistency, could it be good enough reason to use it?
14. Always use text type for all columns
• It is always fun to reimplement date or ip validation in your
code
• You will never mistakenly convert ”12-31-2015 03:01AM” to
”15:01 12 of undef 2015” using text fields
15. Always use improved ”PostgreSQL”
• Postgres is not a perfect database and you are smart
• All that annoying MVCC staff, 32 bit xid and autovacuum
nightmare look like they look because hackers are oldschool
and lazy
• Hack it in a hard way, do not bother yourself with submitting
your patch to the community, just put it into production
• It is easy to maintain such production and keep it compatible
with ”not perfect” PostgreSQL upcoming versions
16. Postgres likes long transactions
• Always call external services from stored procedures (like
sending emails)
16. Postgres likes long transactions
• Always call external services from stored procedures (like
sending emails)
• Oh, it is arguable... It can be, if 100% of developers were
familiar with word timeout
16. Postgres likes long transactions
• Always call external services from stored procedures (like
sending emails)
• Oh, it is arguable... It can be, if 100% of developers were
familiar with word timeout
• Anyway, you can just start transaction and go away for
weekend
17. Load your data to PostgreSQL in a smart manner
• Write your own loader, 100 parallel threads minimum
17. Load your data to PostgreSQL in a smart manner
• Write your own loader, 100 parallel threads minimum
• Never use COPY - it is specially designed for the task
18. Even if you want to backup your database...
• Use replication instead of backup
18. Even if you want to backup your database...
• Use replication instead of backup
• Use pg_dump instead of backup
18. Even if you want to backup your database...
• Use replication instead of backup
• Use pg_dump instead of backup
• Write your own backup script
18. Even if you want to backup your database...
• Use replication instead of backup
• Use pg_dump instead of backup
• Write your own backup script
• As complicated as possible, combine all external tools you
know
18. Even if you want to backup your database...
• Use replication instead of backup
• Use pg_dump instead of backup
• Write your own backup script
• As complicated as possible, combine all external tools you
know
• Never perform a test recovery
Do not forget
That was WORST practice talk
Questions or ideas? Share your story!
ik@postgresql-consulting.com
(I’am preparing this talk to be open sourced)

More Related Content

What's hot

Scylla Summit 2022: Scylla 5.0 New Features, Part 2
Scylla Summit 2022: Scylla 5.0 New Features, Part 2Scylla Summit 2022: Scylla 5.0 New Features, Part 2
Scylla Summit 2022: Scylla 5.0 New Features, Part 2
ScyllaDB
 
Windows IOCP vs Linux EPOLL Performance Comparison
Windows IOCP vs Linux EPOLL Performance ComparisonWindows IOCP vs Linux EPOLL Performance Comparison
Windows IOCP vs Linux EPOLL Performance Comparison
Seungmo Koo
 
MySQL_SQL_Tunning_v0.1.3.docx
MySQL_SQL_Tunning_v0.1.3.docxMySQL_SQL_Tunning_v0.1.3.docx
MySQL_SQL_Tunning_v0.1.3.docx
NeoClova
 
[오픈소스컨설팅]openstack_monitoring_session
[오픈소스컨설팅]openstack_monitoring_session[오픈소스컨설팅]openstack_monitoring_session
[오픈소스컨설팅]openstack_monitoring_session
Chanyeol yoon
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
PostgreSQL-Consulting
 
Oracle-DB: Performance Analysis with Panorama
Oracle-DB: Performance Analysis with PanoramaOracle-DB: Performance Analysis with Panorama
Oracle-DB: Performance Analysis with Panorama
Peter Ramm
 
The PostgreSQL Query Planner
The PostgreSQL Query PlannerThe PostgreSQL Query Planner
The PostgreSQL Query Planner
Command Prompt., Inc
 
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
Ji-Woong Choi
 
PostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized WorldPostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized World
Jignesh Shah
 
Linux Performance Profiling and Monitoring
Linux Performance Profiling and MonitoringLinux Performance Profiling and Monitoring
Linux Performance Profiling and Monitoring
Georg Schönberger
 
Velocity 2017 Performance analysis superpowers with Linux eBPF
Velocity 2017 Performance analysis superpowers with Linux eBPFVelocity 2017 Performance analysis superpowers with Linux eBPF
Velocity 2017 Performance analysis superpowers with Linux eBPF
Brendan Gregg
 
서버학개론(백엔드 서버 개발자를 위한)
서버학개론(백엔드 서버 개발자를 위한)서버학개론(백엔드 서버 개발자를 위한)
서버학개론(백엔드 서버 개발자를 위한)
수보 김
 
[236] 카카오의데이터파이프라인 윤도영
[236] 카카오의데이터파이프라인 윤도영[236] 카카오의데이터파이프라인 윤도영
[236] 카카오의데이터파이프라인 윤도영
NAVER D2
 
MMUG18 - MySQL Failover and Orchestrator
MMUG18 - MySQL Failover and OrchestratorMMUG18 - MySQL Failover and Orchestrator
MMUG18 - MySQL Failover and Orchestrator
Simon J Mudd
 
Machine Learning Model Serving with Backend.AI
Machine Learning Model Serving with Backend.AIMachine Learning Model Serving with Backend.AI
Machine Learning Model Serving with Backend.AI
Jeongkyu Shin
 
Solving PostgreSQL wicked problems
Solving PostgreSQL wicked problemsSolving PostgreSQL wicked problems
Solving PostgreSQL wicked problems
Alexander Korotkov
 
MySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software TestMySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software Test
I Goo Lee
 
Blazing Performance with Flame Graphs
Blazing Performance with Flame GraphsBlazing Performance with Flame Graphs
Blazing Performance with Flame Graphs
Brendan Gregg
 
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
Amazon Web Services Korea
 
MySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn TutorialMySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn Tutorial
Kenny Gryp
 

What's hot (20)

Scylla Summit 2022: Scylla 5.0 New Features, Part 2
Scylla Summit 2022: Scylla 5.0 New Features, Part 2Scylla Summit 2022: Scylla 5.0 New Features, Part 2
Scylla Summit 2022: Scylla 5.0 New Features, Part 2
 
Windows IOCP vs Linux EPOLL Performance Comparison
Windows IOCP vs Linux EPOLL Performance ComparisonWindows IOCP vs Linux EPOLL Performance Comparison
Windows IOCP vs Linux EPOLL Performance Comparison
 
MySQL_SQL_Tunning_v0.1.3.docx
MySQL_SQL_Tunning_v0.1.3.docxMySQL_SQL_Tunning_v0.1.3.docx
MySQL_SQL_Tunning_v0.1.3.docx
 
[오픈소스컨설팅]openstack_monitoring_session
[오픈소스컨설팅]openstack_monitoring_session[오픈소스컨설팅]openstack_monitoring_session
[오픈소스컨설팅]openstack_monitoring_session
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
 
Oracle-DB: Performance Analysis with Panorama
Oracle-DB: Performance Analysis with PanoramaOracle-DB: Performance Analysis with Panorama
Oracle-DB: Performance Analysis with Panorama
 
The PostgreSQL Query Planner
The PostgreSQL Query PlannerThe PostgreSQL Query Planner
The PostgreSQL Query Planner
 
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법
 
PostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized WorldPostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized World
 
Linux Performance Profiling and Monitoring
Linux Performance Profiling and MonitoringLinux Performance Profiling and Monitoring
Linux Performance Profiling and Monitoring
 
Velocity 2017 Performance analysis superpowers with Linux eBPF
Velocity 2017 Performance analysis superpowers with Linux eBPFVelocity 2017 Performance analysis superpowers with Linux eBPF
Velocity 2017 Performance analysis superpowers with Linux eBPF
 
서버학개론(백엔드 서버 개발자를 위한)
서버학개론(백엔드 서버 개발자를 위한)서버학개론(백엔드 서버 개발자를 위한)
서버학개론(백엔드 서버 개발자를 위한)
 
[236] 카카오의데이터파이프라인 윤도영
[236] 카카오의데이터파이프라인 윤도영[236] 카카오의데이터파이프라인 윤도영
[236] 카카오의데이터파이프라인 윤도영
 
MMUG18 - MySQL Failover and Orchestrator
MMUG18 - MySQL Failover and OrchestratorMMUG18 - MySQL Failover and Orchestrator
MMUG18 - MySQL Failover and Orchestrator
 
Machine Learning Model Serving with Backend.AI
Machine Learning Model Serving with Backend.AIMachine Learning Model Serving with Backend.AI
Machine Learning Model Serving with Backend.AI
 
Solving PostgreSQL wicked problems
Solving PostgreSQL wicked problemsSolving PostgreSQL wicked problems
Solving PostgreSQL wicked problems
 
MySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software TestMySQL/MariaDB Proxy Software Test
MySQL/MariaDB Proxy Software Test
 
Blazing Performance with Flame Graphs
Blazing Performance with Flame GraphsBlazing Performance with Flame Graphs
Blazing Performance with Flame Graphs
 
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
PUBG: Battlegrounds 라이브 서비스 EKS 전환 사례 공유 [크래프톤 - 레벨 300] - 발표자: 김정헌, PUBG Dev...
 
MySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn TutorialMySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn Tutorial
 

Similar to PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky

PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
PostgreSQL worst practices, version PGConf.US 2017 by Ilya KosmodemianskyPostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
PostgreSQL-Consulting
 
Unbreaking Your Django Application
Unbreaking Your Django ApplicationUnbreaking Your Django Application
Unbreaking Your Django ApplicationOSCON Byrum
 
Lightening Talk - PostgreSQL Worst Practices
Lightening Talk - PostgreSQL Worst PracticesLightening Talk - PostgreSQL Worst Practices
Lightening Talk - PostgreSQL Worst Practices
PGConf APAC
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
Elizabeth Smith
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
Elizabeth Smith
 
Software + Babies
Software + BabiesSoftware + Babies
Software + Babies
ArangoDB Database
 
Austin Python Learners Meetup - Everything you need to know about programming...
Austin Python Learners Meetup - Everything you need to know about programming...Austin Python Learners Meetup - Everything you need to know about programming...
Austin Python Learners Meetup - Everything you need to know about programming...
Danny Mulligan
 
Lessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’tLessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’t
PGConf APAC
 
Best practices with development of enterprise-scale SharePoint solutions - Pa...
Best practices with development of enterprise-scale SharePoint solutions - Pa...Best practices with development of enterprise-scale SharePoint solutions - Pa...
Best practices with development of enterprise-scale SharePoint solutions - Pa...
SPC Adriatics
 
Keeping MongoDB Data Safe
Keeping MongoDB Data SafeKeeping MongoDB Data Safe
Keeping MongoDB Data Safe
Tony Tam
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
Max De Marzi
 
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users HappyGeek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
IDERA Software
 
Entity framework advanced
Entity framework advancedEntity framework advanced
Entity framework advanced
Usama Nada
 
Building Big Data Streaming Architectures
Building Big Data Streaming ArchitecturesBuilding Big Data Streaming Architectures
Building Big Data Streaming Architectures
David Martínez Rego
 
The 5 Minute MySQL DBA
The 5 Minute MySQL DBAThe 5 Minute MySQL DBA
The 5 Minute MySQL DBA
Irawan Soetomo
 
Generating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in juliaGenerating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in julia
Andre Pemmelaar
 
Internet of Things, TYBSC IT, Semester 5, Unit IV
Internet of Things, TYBSC IT, Semester 5, Unit IVInternet of Things, TYBSC IT, Semester 5, Unit IV
Internet of Things, TYBSC IT, Semester 5, Unit IV
Arti Parab Academics
 
Reading Notes : the practice of programming
Reading Notes : the practice of programmingReading Notes : the practice of programming
Reading Notes : the practice of programming
Juggernaut Liu
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)
Oren Eini
 

Similar to PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky (20)

PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
PostgreSQL worst practices, version PGConf.US 2017 by Ilya KosmodemianskyPostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
PostgreSQL worst practices, version PGConf.US 2017 by Ilya Kosmodemiansky
 
Unbreaking Your Django Application
Unbreaking Your Django ApplicationUnbreaking Your Django Application
Unbreaking Your Django Application
 
Lightening Talk - PostgreSQL Worst Practices
Lightening Talk - PostgreSQL Worst PracticesLightening Talk - PostgreSQL Worst Practices
Lightening Talk - PostgreSQL Worst Practices
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
 
Taming the resource tiger
Taming the resource tigerTaming the resource tiger
Taming the resource tiger
 
Software + Babies
Software + BabiesSoftware + Babies
Software + Babies
 
Austin Python Learners Meetup - Everything you need to know about programming...
Austin Python Learners Meetup - Everything you need to know about programming...Austin Python Learners Meetup - Everything you need to know about programming...
Austin Python Learners Meetup - Everything you need to know about programming...
 
Lessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’tLessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’t
 
Best practices with development of enterprise-scale SharePoint solutions - Pa...
Best practices with development of enterprise-scale SharePoint solutions - Pa...Best practices with development of enterprise-scale SharePoint solutions - Pa...
Best practices with development of enterprise-scale SharePoint solutions - Pa...
 
Keeping MongoDB Data Safe
Keeping MongoDB Data SafeKeeping MongoDB Data Safe
Keeping MongoDB Data Safe
 
Neo4j Training Cypher
Neo4j Training CypherNeo4j Training Cypher
Neo4j Training Cypher
 
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users HappyGeek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
Geek Sync | Top 5 Tips to Keep Always On Always Humming and Users Happy
 
Entity framework advanced
Entity framework advancedEntity framework advanced
Entity framework advanced
 
Gpgpu intro
Gpgpu introGpgpu intro
Gpgpu intro
 
Building Big Data Streaming Architectures
Building Big Data Streaming ArchitecturesBuilding Big Data Streaming Architectures
Building Big Data Streaming Architectures
 
The 5 Minute MySQL DBA
The 5 Minute MySQL DBAThe 5 Minute MySQL DBA
The 5 Minute MySQL DBA
 
Generating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in juliaGenerating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in julia
 
Internet of Things, TYBSC IT, Semester 5, Unit IV
Internet of Things, TYBSC IT, Semester 5, Unit IVInternet of Things, TYBSC IT, Semester 5, Unit IV
Internet of Things, TYBSC IT, Semester 5, Unit IV
 
Reading Notes : the practice of programming
Reading Notes : the practice of programmingReading Notes : the practice of programming
Reading Notes : the practice of programming
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)
 

More from PostgreSQL-Consulting

Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
PostgreSQL-Consulting
 
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
PostgreSQL-Consulting
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016
PostgreSQL-Consulting
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
PostgreSQL-Consulting
 
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQPostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL-Consulting
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015PostgreSQL-Consulting
 
Как PostgreSQL работает с диском
Как PostgreSQL работает с дискомКак PostgreSQL работает с диском
Как PostgreSQL работает с диском
PostgreSQL-Consulting
 
Максим Богук. Postgres-XC
Максим Богук. Postgres-XCМаксим Богук. Postgres-XC
Максим Богук. Postgres-XC
PostgreSQL-Consulting
 
Иван Фролков. Tricky SQL
Иван Фролков. Tricky SQLИван Фролков. Tricky SQL
Иван Фролков. Tricky SQL
PostgreSQL-Consulting
 
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQLИлья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
PostgreSQL-Consulting
 

More from PostgreSQL-Consulting (12)

Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
 
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
Linux IO internals for database administrators (SCaLE 2017 and PGDay Nordic 2...
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
 
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQPostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQ
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
Pgconfru 2015 kosmodemiansky
Pgconfru 2015 kosmodemianskyPgconfru 2015 kosmodemiansky
Pgconfru 2015 kosmodemiansky
 
Как PostgreSQL работает с диском
Как PostgreSQL работает с дискомКак PostgreSQL работает с диском
Как PostgreSQL работает с диском
 
Kosmodemiansky wr 2013
Kosmodemiansky wr 2013Kosmodemiansky wr 2013
Kosmodemiansky wr 2013
 
Максим Богук. Postgres-XC
Максим Богук. Postgres-XCМаксим Богук. Postgres-XC
Максим Богук. Postgres-XC
 
Иван Фролков. Tricky SQL
Иван Фролков. Tricky SQLИван Фролков. Tricky SQL
Иван Фролков. Tricky SQL
 
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQLИлья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
Илья Космодемьянский. Использование очередей асинхронных сообщений с PostgreSQL
 

Recently uploaded

Governing Equations for Fundamental Aerodynamics_Anderson2010.pdf
Governing Equations for Fundamental Aerodynamics_Anderson2010.pdfGoverning Equations for Fundamental Aerodynamics_Anderson2010.pdf
Governing Equations for Fundamental Aerodynamics_Anderson2010.pdf
WENKENLI1
 
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
Amil Baba Dawood bangali
 
CME397 Surface Engineering- Professional Elective
CME397 Surface Engineering- Professional ElectiveCME397 Surface Engineering- Professional Elective
CME397 Surface Engineering- Professional Elective
karthi keyan
 
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
Dr.Costas Sachpazis
 
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
obonagu
 
weather web application report.pdf
weather web application report.pdfweather web application report.pdf
weather web application report.pdf
Pratik Pawar
 
Nuclear Power Economics and Structuring 2024
Nuclear Power Economics and Structuring 2024Nuclear Power Economics and Structuring 2024
Nuclear Power Economics and Structuring 2024
Massimo Talia
 
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTSHeap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Soumen Santra
 
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)
ClaraZara1
 
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdfHybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
fxintegritypublishin
 
Hierarchical Digital Twin of a Naval Power System
Hierarchical Digital Twin of a Naval Power SystemHierarchical Digital Twin of a Naval Power System
Hierarchical Digital Twin of a Naval Power System
Kerry Sado
 
DESIGN A COTTON SEED SEPARATION MACHINE.docx
DESIGN A COTTON SEED SEPARATION MACHINE.docxDESIGN A COTTON SEED SEPARATION MACHINE.docx
DESIGN A COTTON SEED SEPARATION MACHINE.docx
FluxPrime1
 
Forklift Classes Overview by Intella Parts
Forklift Classes Overview by Intella PartsForklift Classes Overview by Intella Parts
Forklift Classes Overview by Intella Parts
Intella Parts
 
Student information management system project report ii.pdf
Student information management system project report ii.pdfStudent information management system project report ii.pdf
Student information management system project report ii.pdf
Kamal Acharya
 
AP LAB PPT.pdf ap lab ppt no title specific
AP LAB PPT.pdf ap lab ppt no title specificAP LAB PPT.pdf ap lab ppt no title specific
AP LAB PPT.pdf ap lab ppt no title specific
BrazilAccount1
 
Basic Industrial Engineering terms for apparel
Basic Industrial Engineering terms for apparelBasic Industrial Engineering terms for apparel
Basic Industrial Engineering terms for apparel
top1002
 
Railway Signalling Principles Edition 3.pdf
Railway Signalling Principles Edition 3.pdfRailway Signalling Principles Edition 3.pdf
Railway Signalling Principles Edition 3.pdf
TeeVichai
 
road safety engineering r s e unit 3.pdf
road safety engineering  r s e unit 3.pdfroad safety engineering  r s e unit 3.pdf
road safety engineering r s e unit 3.pdf
VENKATESHvenky89705
 
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
MdTanvirMahtab2
 
Water Industry Process Automation and Control Monthly - May 2024.pdf
Water Industry Process Automation and Control Monthly - May 2024.pdfWater Industry Process Automation and Control Monthly - May 2024.pdf
Water Industry Process Automation and Control Monthly - May 2024.pdf
Water Industry Process Automation & Control
 

Recently uploaded (20)

Governing Equations for Fundamental Aerodynamics_Anderson2010.pdf
Governing Equations for Fundamental Aerodynamics_Anderson2010.pdfGoverning Equations for Fundamental Aerodynamics_Anderson2010.pdf
Governing Equations for Fundamental Aerodynamics_Anderson2010.pdf
 
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...
 
CME397 Surface Engineering- Professional Elective
CME397 Surface Engineering- Professional ElectiveCME397 Surface Engineering- Professional Elective
CME397 Surface Engineering- Professional Elective
 
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...
 
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
 
weather web application report.pdf
weather web application report.pdfweather web application report.pdf
weather web application report.pdf
 
Nuclear Power Economics and Structuring 2024
Nuclear Power Economics and Structuring 2024Nuclear Power Economics and Structuring 2024
Nuclear Power Economics and Structuring 2024
 
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTSHeap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
 
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)
 
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdfHybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdf
 
Hierarchical Digital Twin of a Naval Power System
Hierarchical Digital Twin of a Naval Power SystemHierarchical Digital Twin of a Naval Power System
Hierarchical Digital Twin of a Naval Power System
 
DESIGN A COTTON SEED SEPARATION MACHINE.docx
DESIGN A COTTON SEED SEPARATION MACHINE.docxDESIGN A COTTON SEED SEPARATION MACHINE.docx
DESIGN A COTTON SEED SEPARATION MACHINE.docx
 
Forklift Classes Overview by Intella Parts
Forklift Classes Overview by Intella PartsForklift Classes Overview by Intella Parts
Forklift Classes Overview by Intella Parts
 
Student information management system project report ii.pdf
Student information management system project report ii.pdfStudent information management system project report ii.pdf
Student information management system project report ii.pdf
 
AP LAB PPT.pdf ap lab ppt no title specific
AP LAB PPT.pdf ap lab ppt no title specificAP LAB PPT.pdf ap lab ppt no title specific
AP LAB PPT.pdf ap lab ppt no title specific
 
Basic Industrial Engineering terms for apparel
Basic Industrial Engineering terms for apparelBasic Industrial Engineering terms for apparel
Basic Industrial Engineering terms for apparel
 
Railway Signalling Principles Edition 3.pdf
Railway Signalling Principles Edition 3.pdfRailway Signalling Principles Edition 3.pdf
Railway Signalling Principles Edition 3.pdf
 
road safety engineering r s e unit 3.pdf
road safety engineering  r s e unit 3.pdfroad safety engineering  r s e unit 3.pdf
road safety engineering r s e unit 3.pdf
 
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)
 
Water Industry Process Automation and Control Monthly - May 2024.pdf
Water Industry Process Automation and Control Monthly - May 2024.pdfWater Industry Process Automation and Control Monthly - May 2024.pdf
Water Industry Process Automation and Control Monthly - May 2024.pdf
 

PostgreSQL worst practices, version FOSDEM PGDay 2017 by Ilya Kosmodemiansky

  • 1. PostgreSQL worst practices at FOSDEM PGDay Brussels 2017 Ilya Kosmodemiansky ik@postgresql-consulting.com
  • 2. Best practices are just boring • Never follow them, try worst practices • Only those practices can really help you to screw the things up most effectively • PostgreSQL consultants are nice people, so try to make them happy
  • 3. How it works? • I have a list, a little bit more than 100 worst practices • I do not make this stuff up, all of them are real-life examples • I reshuffle my list every time before presenting and extract some amount of examples • Well, there are some things, which I like more or less, so it is not a very honest shuffle
  • 4. 0. Do not use indexes (a test one!) • Basically, there is no difference between full table scan and index scan • You can check that. Just insert 10 rows into a test table on your test server and compare. • Nobody deals with more than 10 row tables in production!
  • 5. 1. Use ORM • All databases share the same syntax • You must write database-independent code • Are there any benefits, which are based on database specific features? • It always good to learn a new complicated technology
  • 6. 2. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level
  • 7. 2. Move joins to your application • Just select * a couple of tables into the application written in your favorite programming language • Than join them at the application level • Now you only need to implement nested loop join, hash join and merge join as well as query optimizer and page cache
  • 8. 3. Be in trend, be schema-less • You do not need to design the schema • You need only one table, two columns: id bigserial and extra jsonb • JSONB datatype is pretty effective in PostgreSQL, you can search in it just like in a well-structured table • Even if you put a 100M of JSON in it • Even if you have 1000+ tps
  • 9. 4. Be agile, use EAV • You need only 3 tables: entity, attribute, value
  • 10. 4. Be agile, use EAV • You need only 3 tables: entity, attribute, value • At some point add the 4th: attribute_type
  • 11. 4. Be agile, use EAV • You need only 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • Whet it starts to work slow, just call those four tables The Core and add 1000+ tables with denormalized data
  • 12. 4. Be agile, use EAV • You need only 3 tables: entity, attribute, value • At some point add the 4th: attribute_type • Whet it starts to work slow, just call those four tables The Core and add 1000+ tables with denormalized data • If it is not enough, you can always add value_version
  • 13. 5. Try to create as many indexes as you can • Indexes consume no disk space • Indexes consume no shared_bufers • There is no overhead on DML if one and every column in a table covered with bunch of indexes • Optimizer will definitely choose your index once you created it • Keep calm and create more indexes
  • 14. 6. Always keep all your time series data • Time series data like tables with logs or session history should be never deleted, aggregated or archived, you always need to keep it all
  • 15. 6. Always keep all your time series data • Time series data like tables with logs or session history should be never deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space
  • 16. 6. Always keep all your time series data • Time series data like tables with logs or session history should be never deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data
  • 17. 6. Always keep all your time series data • Time series data like tables with logs or session history should be never deleted, aggregated or archived, you always need to keep it all • You will always know where to check, if you run out of disk space • You can always call that Big Data • Solve the problem using partitioning... one partition for an hour or for a minute
  • 18. 7. Turn autovacuum off • It is quite auxiliary process, you can easily stop it • There is no problem at all to have 100Gb data in a database which is 1Tb in size • 2-3Tb RAM servers are cheap, IO is a fastest thing in modern computing • Besides of that, everyone likes BigData
  • 19. 8. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover
  • 20. 8. Keep master and slave on different hardware • That will maximize the possibility of unsuccessful failover • To make things worser, you can change only slave-related parameters at slave, leaving defaults for shared_buffers etc.
  • 21. 9. Put a synchronous replica to remote DC • Indeed! That will maximize availability!
  • 22. 9. Put a synchronous replica to remote DC • Indeed! That will maximize availability! • Especially, if you put the replica to another continent
  • 23. 10. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch
  • 24. 10. Reinvent Slony • If you need some data replication to another database, try to implement it from scratch • That allows you to run into all problems, PostgreSQL have had since introducing Slony
  • 25. 11. Use as many count(*) as you can • Figure 301083021830123921 is very informative for the end user • If it changes in a second to 30108302894839434020, it is still informative • select count(*) from sometable is a quite light-weighted query • Tuple estimation from pg_catalog can never be precise enough for you
  • 26. 12. Never use graphical monitoring • You do not need graphs • Because it is an easy task to guess what was happened yesterday at 2 a.m. using command line and grep only
  • 27. 13. Never use Foreign Keys (Use local produced instead!) • Consistency control at application level always works as expected • You will never get data inconsistency without constraints • Even if you already have a bullet proof framework to maintain consistency, could it be good enough reason to use it?
  • 28. 14. Always use text type for all columns • It is always fun to reimplement date or ip validation in your code • You will never mistakenly convert ”12-31-2015 03:01AM” to ”15:01 12 of undef 2015” using text fields
  • 29. 15. Always use improved ”PostgreSQL” • Postgres is not a perfect database and you are smart • All that annoying MVCC staff, 32 bit xid and autovacuum nightmare look like they look because hackers are oldschool and lazy • Hack it in a hard way, do not bother yourself with submitting your patch to the community, just put it into production • It is easy to maintain such production and keep it compatible with ”not perfect” PostgreSQL upcoming versions
  • 30. 16. Postgres likes long transactions • Always call external services from stored procedures (like sending emails)
  • 31. 16. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout
  • 32. 16. Postgres likes long transactions • Always call external services from stored procedures (like sending emails) • Oh, it is arguable... It can be, if 100% of developers were familiar with word timeout • Anyway, you can just start transaction and go away for weekend
  • 33. 17. Load your data to PostgreSQL in a smart manner • Write your own loader, 100 parallel threads minimum
  • 34. 17. Load your data to PostgreSQL in a smart manner • Write your own loader, 100 parallel threads minimum • Never use COPY - it is specially designed for the task
  • 35. 18. Even if you want to backup your database... • Use replication instead of backup
  • 36. 18. Even if you want to backup your database... • Use replication instead of backup • Use pg_dump instead of backup
  • 37. 18. Even if you want to backup your database... • Use replication instead of backup • Use pg_dump instead of backup • Write your own backup script
  • 38. 18. Even if you want to backup your database... • Use replication instead of backup • Use pg_dump instead of backup • Write your own backup script • As complicated as possible, combine all external tools you know
  • 39. 18. Even if you want to backup your database... • Use replication instead of backup • Use pg_dump instead of backup • Write your own backup script • As complicated as possible, combine all external tools you know • Never perform a test recovery
  • 40. Do not forget That was WORST practice talk
  • 41. Questions or ideas? Share your story! ik@postgresql-consulting.com (I’am preparing this talk to be open sourced)