SlideShare a Scribd company logo
Tuning Linux for your
database
Colin Charles,Team MariaDB, MariaDB Corporation
colin@mariadb.com / byte@bytebot.net
http://bytebot.net/blog/ | @bytebot on Twitter
FLOSSUK 2016, London
16 March 2016
1
whoami
• Work on MariaDB at MariaDB Corporation
(SkySQL Ab)
• Merged with Monty Program Ab, makers of
MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO),
OpenOffice.org
• MySQL Community Contributor of theYear Award
winner 2014
2
Agenda
• Hardware or cloud
• I/O
• Filesystems
• Memory
• CPU
• Network
• Resources
3
Operating System
• Focus of this talk & tools: Linux
• But distributions make a difference too
• like RHEL6 has tuned but Ubuntu doesn’t
• Distribution versions can make a difference
too
• RHEL5: ktune; RHEL6: tuned; RHEL7:
tuned throughput-performance
4
Servers
• When you buy servers... do you think in
terms of CPU, then memory, then I/O?
• Database servers clearly love I/O more than
memory more than CPU
• queries! scans of large tables? index scans
with random I/O? BI?
• Durability requires each transaction commit
to be flushed to disk - i.e. call fsync()
5
I/O
• What kind of storage do you use?
• SAS or SATA?
• Ethernet (NAS, DRBD)
• SSD
• Fusion IO (NVMFS)
6
Schedulers (I/O
Elevators)
• cfq: default, great for slower storage
(SATA)
• noop: low latency storage (SSD)
• deadline: multi-process apps
• Most database workloads benefit from the
deadline scheduler (goal is after all to
minimise seeks, prioritise process I/O)
7
Elevators continued
• Boot time: elevator=deadline
•echo “deadline” > /sys/class/
block/sdaN/queue/scheduler
• http://dimitrik.free.fr/blog/archives/2012/01/
mysql-performance-linux-io.html
• http://www.mysqlperformanceblog.com/
2009/01/30/linux-schedulers-in-tpcc-like-
benchmark/
8
Sample elevator results
9
1 thread
tps
1 thread
ioutil%
8 threads
tps
8 threads
ioutil%
deadline* 15000 <1% 19100 <1%
deadline 6850 32% 15250 45%
cfq 5880 45% 1240 94%
File systems
• Choices: ext4, XFS
• mount options:
• ext4: rw,
nosuid,
noatime,
data=ordered,
nobarrier
• xfs: nobarrier
• Turn off I/O barriers
• http://lwn.net/Articles/
283161/
• Separate files (data/logs/
undo/etc.) or not?
• Don’t forget flash backed
write caches or battery
backed write cache on
RAID cards (way more
fsync/s)
• iostat -dmx <interval> is
your friend
10
SSD & Flash
• SSD
• much more IOPS than traditional disks
• lower latency
• make sure you have a new RAID controller
• Flash
• more IOPS than SSD
• very low latency
• InnoDB doublewrite buffer -- MariaDB 10.0, Percona
Server 5.6 (Fusion IO), MySQL 5.7
11
RAID
• RAID0 - fast writes, reads, no redundancy
• RAID1 - slower writes, fast reads
• RAID5 - slow for random writes, fast for
sequential writes, fast reads too but slow
recovery
• RAID10 - fast reads & writes
12
LVM
• Why?
• easily expand disk
• snapshot backups
• Why not?
• 2-3% performance penalty
• Snapshot penalties
13
Memory
• The more RAM the better -> reduces I/O requirements
• Use ECC RAM
• NUMA - “swap insanity”
• http://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-
numa-architecture/
•numactl --interleave=all mysqld &
• Twitter patch/Percona Server 5.6
• numa_interleave option
• innodb_buffer_pool_populate - NUMA decisions
to be made when buffer cache is clean
14
Why does MySQL need
memory?
• Per-session buffers
• sort buffer, temp tables
• Metadata/locking
• index statistics, table definitions
• Caching data
• decrease reads: faster response time
• decrease random writes: queues write in cache;
perform more sequential writes
15
Where is memory
used?
• filesystem cache
• binary / relay logs
• MyISAM data relies on filesystem cache
• MySQL cache:
• innodb_buffer_pool_size (pages), MyISAM’s
key_buffer_size (indexes)
• Per session buffers
•sort_buffer_size, join_buffer_size,
read_buffer_size, tmp_table_size
•free -m / SHOW ENGINE INNODB STATUS
16
Huge pages
• 2M (4M even) pages vs 4K standard Linux
page
•vm.nr_hugepages=8192
• MySQL benefits from hugepages, MongoDB
doesn’t
17
Swappiness
• Controls how aggressively the system reclaims
“mapped” memory
• default: 60, vm.swappiness=0 is ok (but not for newer
kernels!)
• decrease: aggressive reclaiming of unmapped
pagecache memory
• increase: aggressive swapping of mapped memory
• 3.5 kernels change behaviour (and this was
backported to RHEL 2.6.32-303)
18
CPU
• default is ondemand, you should turn it to
performance generally
•echo "performance" > /sys/devices/
system/cpu/cpu0/cpufreq/
scaling_governor
• disable powersave mode
• cpufrequtils is where it’s at
• look at BIOS: turbo mode/C-state/OS power control
19
Network
• Tools: tc, dropwatch
• Use gigabit Ethernet (more bandwidth/
throughput, less latency)
• Can also trunk/bond for HA
20
Network II
• ARP filter - prevents ARP flux
•echo 1 > /proc/sys/net/ipv4/
conf/all/arp_filter
• MTU size at 9,000 - “jumbo frames”
• sar -n DEV
21
Network III
• allow more connections to queue up: echo 4096 > /
proc/sys/net/ipv4/tcp_max_syn_backlog
• /etc/sysctl.conf -
net.ipv4.tcp_max_syn_backlog = 4096
• increase kernel packet buffer:
net.core.netdev_max_backlog = 4096
• increase socket connection wait queue:
net.core.somaxconn = 4096
• Reduce TCP timeout that comes after closing socket (default
1m): net.ipv4.tcp_fin_timeout = 30
22
Network IV
• Fully synchronous replication, ala, Galera
Cluster? (NDBCluster too)
• Fastest as your slowest node
• evs.send_window + evs.user_send_window
(something large e.g. 512)
23
Resource Limits
• ulimits / /etc/security/limits.conf - they need to be
high/sensible over the defaults
• -f (file size): unlimited
• -t (cpu time): unlimited
• -v (virtmem): unlimited
• -n (open files): 64000
• -m (memsize): unlimited
• -u (processes/threads): 32000
24
Tune that database
• Focus on database design - find slow, crappy
queries and fix them (eg. mysql: pt-
query-digest)
• Reduce locking or time waiting - always
benchmark your application continually
• Don’t forget /etc/my.cnf (and appropriate
friends)
25
KVM/Xen/Virtualization
• Most of your raw database performance
comes from being hardware-centric
• KVM at device level you can turn off caching
• Native AIO used over threaded nowadays
• VMWare: http://kb.vmware.com/selfservice/
microsites/search.do?
language=en_US&cmd=displayKC&externalI
d=1008542
26
Containers
• Kubernetes - vitess.io rolls out w/o issue,
CoreOS, Docker
• Remember when benchmarking, you have
to look at local vs remote benchmarking
(networking can add overhead)
• 1 thread (up to 25% difference), but average
seems to be 15% loss for r/w workloads;
remotely? 32% is possible
27
EC2
• EC2 - instance limitations
• EBS - unpredictable I/O performance, use
RAID10 or RAID0
• RDS - similar performance between EBS
RAID10 MySQL & RDS
• choice of MySQL 5.7 (and older) or
MariaDB Server 10.0 or PostgreSQL
28
Why MariaDB/Percona
Server
• Threadpool
• NUMA interleaving
• Numerous performance fixes in XtraDB
like fast InnoDB restarts, buffer pool
warming, etc.
• Can run (supported) MariaDB Galera
Cluster/Percona XtraDB Cluster
29
Hardware overall
• Test everything
• Sometimes NIC’s are a bad batch or the
driver is buggy (RHEL5 + some Broadcom
NICs drop packets under high load - bnx2
before 2.6.33)
• Sometimes you get a bad batch of disk
30
Hadoop/Hbase
• running this in EC2 is problematic
• make sure name resolution works
• enable bonding
• networking: hadoop uses big buffers (64MB blocks by default)
• net.ipv4.tcp_rmem = 32768 436600 4194304
• net.ipv4.tcp_wmem = 32768 436600 4194304
• http://docs.hortonworks.com/HDPDocuments/HDP2/
HDP-2.1.3/bk_cluster-planning-guide/content/ch_hardware-
recommendations.html
31
Other databases
• MongoDB: http://docs.mongodb.org/manual/
administration/production-notes/
• PostgreSQL: http://wiki.postgresql.org/wiki/
Tuning_Your_PostgreSQL_Server
• Neo4j: http://docs.neo4j.org/chunked/stable/
configuration-linux-notes.html
• Riak: http://docs.basho.com/riak/1.4.0/
cookbooks/Linux-Performance-Tuning/
32
SELinux, firewalls,
security
• The easy way is to “disable SELinux”
• The reality is you should learn it
• Its just like a firewall that you really should
configure
33
Tools
• ps
• vmstat 1
• iostat
• top
• free
• sar (sar -n DEV 1)
• gdb
• tcpdump
• strace
• oprofile
• htop
• MySQL only: Percona
Toolkit (pt-diskstats, pt-
summary, pt-query-digest,
etc.)
34
Benchmarking
• sysbench
• OLTP test, use
tables with 20M
rows and 20M
transactions, check
1-128 threads/run
• LinkBench
• Yahoo! Cloud Serving
Benchmark
• https://github.com/
brianfrankcooper/
YCSB
• Google’s PerfKit
Benchmarker
• https://github.com/
GoogleCloudPlatfor
m/
PerfKitBenchmarke
r
35
Books
36
Resources
• http://doc.opensuse.org/products/draft/SLES/
SLES-tuning_sd_draft/book.sle.tuning.html
• https://access.redhat.com/site/documentation/
en-US/Red_Hat_Enterprise_Linux/6/html-single/
Performance_Tuning_Guide/index.html
• https://access.redhat.com/site/documentation/
en-US/Red_Hat_Enterprise_Linux/6/html-single/
Virtualization_Tuning_and_Optimization_Guide
/index.html
37
Thanks/Q&A
Colin Charles, colin@mariadb.com | byte@bytebot.net
http://bytebot.net/blog/ | @bytebot
slides: slideshare.net/bytebot/
Download MariaDB and give it a try: http://mariadb.org/
38

More Related Content

What's hot

Better encryption & security with MariaDB 10.1 & MySQL 5.7
Better encryption & security with MariaDB 10.1 & MySQL 5.7Better encryption & security with MariaDB 10.1 & MySQL 5.7
Better encryption & security with MariaDB 10.1 & MySQL 5.7
Colin Charles
 
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB MeetupMariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
Colin Charles
 
My first moments with MongoDB
My first moments with MongoDBMy first moments with MongoDB
My first moments with MongoDB
Colin Charles
 
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityBest practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Colin Charles
 
Cool MariaDB Plugins
Cool MariaDB Plugins Cool MariaDB Plugins
Cool MariaDB Plugins
Colin Charles
 
Securing your MySQL / MariaDB Server data
Securing your MySQL / MariaDB Server dataSecuring your MySQL / MariaDB Server data
Securing your MySQL / MariaDB Server data
Colin Charles
 
Lessons from database failures
Lessons from database failuresLessons from database failures
Lessons from database failures
Colin Charles
 
A beginners guide to MariaDB
A beginners guide to MariaDBA beginners guide to MariaDB
A beginners guide to MariaDB
Colin Charles
 
MariaDB 10 Tutorial - 13.11.11 - Percona Live London
MariaDB 10 Tutorial - 13.11.11 - Percona Live LondonMariaDB 10 Tutorial - 13.11.11 - Percona Live London
MariaDB 10 Tutorial - 13.11.11 - Percona Live London
Ivan Zoratti
 
MariaDB 10: The Complete Tutorial
MariaDB 10: The Complete TutorialMariaDB 10: The Complete Tutorial
MariaDB 10: The Complete Tutorial
Colin Charles
 
The Complete MariaDB Server tutorial
The Complete MariaDB Server tutorialThe Complete MariaDB Server tutorial
The Complete MariaDB Server tutorial
Colin Charles
 
Why MariaDB?
Why MariaDB?Why MariaDB?
Why MariaDB?
Colin Charles
 
MariaDB 10 and what's new with the project
MariaDB 10 and what's new with the projectMariaDB 10 and what's new with the project
MariaDB 10 and what's new with the project
Colin Charles
 
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScaleThe Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
Colin Charles
 
MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC
Colin Charles
 
MariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQLMariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQL
Colin Charles
 
MySQL features missing in MariaDB Server
MySQL features missing in MariaDB ServerMySQL features missing in MariaDB Server
MySQL features missing in MariaDB Server
Colin Charles
 
MariaDB: in-depth (hands on training in Seoul)
MariaDB: in-depth (hands on training in Seoul)MariaDB: in-depth (hands on training in Seoul)
MariaDB: in-depth (hands on training in Seoul)
Colin Charles
 
Lessons from database failures
Lessons from database failures Lessons from database failures
Lessons from database failures
Colin Charles
 
The Complete MariaDB Server Tutorial - Percona Live 2015
The Complete MariaDB Server Tutorial - Percona Live 2015The Complete MariaDB Server Tutorial - Percona Live 2015
The Complete MariaDB Server Tutorial - Percona Live 2015
Colin Charles
 

What's hot (20)

Better encryption & security with MariaDB 10.1 & MySQL 5.7
Better encryption & security with MariaDB 10.1 & MySQL 5.7Better encryption & security with MariaDB 10.1 & MySQL 5.7
Better encryption & security with MariaDB 10.1 & MySQL 5.7
 
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB MeetupMariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
 
My first moments with MongoDB
My first moments with MongoDBMy first moments with MongoDB
My first moments with MongoDB
 
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityBest practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High Availability
 
Cool MariaDB Plugins
Cool MariaDB Plugins Cool MariaDB Plugins
Cool MariaDB Plugins
 
Securing your MySQL / MariaDB Server data
Securing your MySQL / MariaDB Server dataSecuring your MySQL / MariaDB Server data
Securing your MySQL / MariaDB Server data
 
Lessons from database failures
Lessons from database failuresLessons from database failures
Lessons from database failures
 
A beginners guide to MariaDB
A beginners guide to MariaDBA beginners guide to MariaDB
A beginners guide to MariaDB
 
MariaDB 10 Tutorial - 13.11.11 - Percona Live London
MariaDB 10 Tutorial - 13.11.11 - Percona Live LondonMariaDB 10 Tutorial - 13.11.11 - Percona Live London
MariaDB 10 Tutorial - 13.11.11 - Percona Live London
 
MariaDB 10: The Complete Tutorial
MariaDB 10: The Complete TutorialMariaDB 10: The Complete Tutorial
MariaDB 10: The Complete Tutorial
 
The Complete MariaDB Server tutorial
The Complete MariaDB Server tutorialThe Complete MariaDB Server tutorial
The Complete MariaDB Server tutorial
 
Why MariaDB?
Why MariaDB?Why MariaDB?
Why MariaDB?
 
MariaDB 10 and what's new with the project
MariaDB 10 and what's new with the projectMariaDB 10 and what's new with the project
MariaDB 10 and what's new with the project
 
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScaleThe Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
 
MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC
 
MariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQLMariaDB Server Compatibility with MySQL
MariaDB Server Compatibility with MySQL
 
MySQL features missing in MariaDB Server
MySQL features missing in MariaDB ServerMySQL features missing in MariaDB Server
MySQL features missing in MariaDB Server
 
MariaDB: in-depth (hands on training in Seoul)
MariaDB: in-depth (hands on training in Seoul)MariaDB: in-depth (hands on training in Seoul)
MariaDB: in-depth (hands on training in Seoul)
 
Lessons from database failures
Lessons from database failures Lessons from database failures
Lessons from database failures
 
The Complete MariaDB Server Tutorial - Percona Live 2015
The Complete MariaDB Server Tutorial - Percona Live 2015The Complete MariaDB Server Tutorial - Percona Live 2015
The Complete MariaDB Server Tutorial - Percona Live 2015
 

Viewers also liked

Streaming replication in practice
Streaming replication in practiceStreaming replication in practice
Streaming replication in practice
Alexey Lesovsky
 
GitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedGitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons Learned
Alexey Lesovsky
 
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
Андрей Шорин
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
PostgreSQL-Consulting
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
PostgreSQL-Consulting
 
Troubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming ReplicationTroubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming Replication
Alexey Lesovsky
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
Alexey Lesovsky
 
MariaDB and Cassandra Interoperability
MariaDB and Cassandra InteroperabilityMariaDB and Cassandra Interoperability
MariaDB and Cassandra Interoperability
Colin Charles
 
Opslag van long tail producten in e-warehouse
Opslag van long tail producten in e-warehouseOpslag van long tail producten in e-warehouse
Opslag van long tail producten in e-warehouse
Storeganizer
 
Performance Tuning a Cloud Application: A Real World Case Study
Performance Tuning a Cloud Application: A Real World Case StudyPerformance Tuning a Cloud Application: A Real World Case Study
Performance Tuning a Cloud Application: A Real World Case Study
shane_gibson
 
Material Auxiliar Para Curso BáSico Msp430 1 A 54
Material Auxiliar Para Curso BáSico Msp430   1 A 54Material Auxiliar Para Curso BáSico Msp430   1 A 54
Material Auxiliar Para Curso BáSico Msp430 1 A 54
Texas Instruments
 
Yahoo: Experiences with MySQL GTID and Multi Threaded Replication
Yahoo: Experiences with MySQL GTID and Multi Threaded ReplicationYahoo: Experiences with MySQL GTID and Multi Threaded Replication
Yahoo: Experiences with MySQL GTID and Multi Threaded Replication
Yashada Jadhav
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
Alexey Lesovsky
 
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQPostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL-Consulting
 
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
Alexey Lesovsky
 
Linux NUMA & Databases: Perils and Opportunities
Linux NUMA & Databases: Perils and OpportunitiesLinux NUMA & Databases: Perils and Opportunities
Linux NUMA & Databases: Perils and Opportunities
Raghavendra Prabhu
 
Lessons from {distributed,remote,virtual} communities and companies
Lessons from {distributed,remote,virtual} communities and companiesLessons from {distributed,remote,virtual} communities and companies
Lessons from {distributed,remote,virtual} communities and companies
Colin Charles
 

Viewers also liked (17)

Streaming replication in practice
Streaming replication in practiceStreaming replication in practice
Streaming replication in practice
 
GitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedGitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons Learned
 
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
Как HeadHunter удалось безопасно нарушить RFC 793 (TCP) и обойти сетевые лову...
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
Troubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming ReplicationTroubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming Replication
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
 
MariaDB and Cassandra Interoperability
MariaDB and Cassandra InteroperabilityMariaDB and Cassandra Interoperability
MariaDB and Cassandra Interoperability
 
Opslag van long tail producten in e-warehouse
Opslag van long tail producten in e-warehouseOpslag van long tail producten in e-warehouse
Opslag van long tail producten in e-warehouse
 
Performance Tuning a Cloud Application: A Real World Case Study
Performance Tuning a Cloud Application: A Real World Case StudyPerformance Tuning a Cloud Application: A Real World Case Study
Performance Tuning a Cloud Application: A Real World Case Study
 
Material Auxiliar Para Curso BáSico Msp430 1 A 54
Material Auxiliar Para Curso BáSico Msp430   1 A 54Material Auxiliar Para Curso BáSico Msp430   1 A 54
Material Auxiliar Para Curso BáSico Msp430 1 A 54
 
Yahoo: Experiences with MySQL GTID and Multi Threaded Replication
Yahoo: Experiences with MySQL GTID and Multi Threaded ReplicationYahoo: Experiences with MySQL GTID and Multi Threaded Replication
Yahoo: Experiences with MySQL GTID and Multi Threaded Replication
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.
 
PostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQPostgreSQL Meetup Berlin at Zalando HQ
PostgreSQL Meetup Berlin at Zalando HQ
 
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
 
Linux NUMA & Databases: Perils and Opportunities
Linux NUMA & Databases: Perils and OpportunitiesLinux NUMA & Databases: Perils and Opportunities
Linux NUMA & Databases: Perils and Opportunities
 
Lessons from {distributed,remote,virtual} communities and companies
Lessons from {distributed,remote,virtual} communities and companiesLessons from {distributed,remote,virtual} communities and companies
Lessons from {distributed,remote,virtual} communities and companies
 

Similar to Tuning Linux for your database FLOSSUK 2016

OSDC 2016 - Tuning Linux for your Database by Colin Charles
OSDC 2016 - Tuning Linux for your Database by Colin CharlesOSDC 2016 - Tuning Linux for your Database by Colin Charles
OSDC 2016 - Tuning Linux for your Database by Colin Charles
NETWAYS
 
Running MySQL on Linux
Running MySQL on LinuxRunning MySQL on Linux
Running MySQL on Linux
Great Wide Open
 
Tuning Linux for MongoDB
Tuning Linux for MongoDBTuning Linux for MongoDB
Tuning Linux for MongoDB
Tim Vaillancourt
 
MySQL Performance Tuning London Meetup June 2017
MySQL Performance Tuning London Meetup June 2017MySQL Performance Tuning London Meetup June 2017
MySQL Performance Tuning London Meetup June 2017
Ivan Zoratti
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User Group
HungWei Chiu
 
Tuning linux for mongo db
Tuning linux for mongo dbTuning linux for mongo db
Tuning linux for mongo db
Soumya Bhattacharyya
 
Windows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined StorageWindows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined Storage
Aidan Finn
 
Introduction to TokuDB v7.5 and Read Free Replication
Introduction to TokuDB v7.5 and Read Free ReplicationIntroduction to TokuDB v7.5 and Read Free Replication
Introduction to TokuDB v7.5 and Read Free Replication
Tim Callaghan
 
Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis
PyData
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
Mike Pittaro
 
pps Matters
pps Matterspps Matters
CPU Caches
CPU CachesCPU Caches
CPU Caches
shinolajla
 
Oracle Performance On Linux X86 systems
Oracle  Performance On Linux  X86 systems Oracle  Performance On Linux  X86 systems
Oracle Performance On Linux X86 systems
Baruch Osoveskiy
 
Deployment Strategies (Mongo Austin)
Deployment Strategies (Mongo Austin)Deployment Strategies (Mongo Austin)
Deployment Strategies (Mongo Austin)
MongoDB
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_eng
DaeMyung Kang
 
Colvin exadata mistakes_ioug_2014
Colvin exadata mistakes_ioug_2014Colvin exadata mistakes_ioug_2014
Colvin exadata mistakes_ioug_2014
marvin herrera
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
Mike Pittaro
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
odsc
 
In-memory Caching in HDFS: Lower Latency, Same Great Taste
In-memory Caching in HDFS: Lower Latency, Same Great TasteIn-memory Caching in HDFS: Lower Latency, Same Great Taste
In-memory Caching in HDFS: Lower Latency, Same Great Taste
DataWorks Summit
 
Drupal performance
Drupal performanceDrupal performance
Drupal performance
Piyuesh Kumar
 

Similar to Tuning Linux for your database FLOSSUK 2016 (20)

OSDC 2016 - Tuning Linux for your Database by Colin Charles
OSDC 2016 - Tuning Linux for your Database by Colin CharlesOSDC 2016 - Tuning Linux for your Database by Colin Charles
OSDC 2016 - Tuning Linux for your Database by Colin Charles
 
Running MySQL on Linux
Running MySQL on LinuxRunning MySQL on Linux
Running MySQL on Linux
 
Tuning Linux for MongoDB
Tuning Linux for MongoDBTuning Linux for MongoDB
Tuning Linux for MongoDB
 
MySQL Performance Tuning London Meetup June 2017
MySQL Performance Tuning London Meetup June 2017MySQL Performance Tuning London Meetup June 2017
MySQL Performance Tuning London Meetup June 2017
 
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User GroupHigh performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User Group
 
Tuning linux for mongo db
Tuning linux for mongo dbTuning linux for mongo db
Tuning linux for mongo db
 
Windows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined StorageWindows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined Storage
 
Introduction to TokuDB v7.5 and Read Free Replication
Introduction to TokuDB v7.5 and Read Free ReplicationIntroduction to TokuDB v7.5 and Read Free Replication
Introduction to TokuDB v7.5 and Read Free Replication
 
Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis Mike Pittaro - High Performance Hardware for Data Analysis
Mike Pittaro - High Performance Hardware for Data Analysis
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
pps Matters
pps Matterspps Matters
pps Matters
 
CPU Caches
CPU CachesCPU Caches
CPU Caches
 
Oracle Performance On Linux X86 systems
Oracle  Performance On Linux  X86 systems Oracle  Performance On Linux  X86 systems
Oracle Performance On Linux X86 systems
 
Deployment Strategies (Mongo Austin)
Deployment Strategies (Mongo Austin)Deployment Strategies (Mongo Austin)
Deployment Strategies (Mongo Austin)
 
Redis trouble shooting_eng
Redis trouble shooting_engRedis trouble shooting_eng
Redis trouble shooting_eng
 
Colvin exadata mistakes_ioug_2014
Colvin exadata mistakes_ioug_2014Colvin exadata mistakes_ioug_2014
Colvin exadata mistakes_ioug_2014
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
High Performance Hardware for Data Analysis
High Performance Hardware for Data AnalysisHigh Performance Hardware for Data Analysis
High Performance Hardware for Data Analysis
 
In-memory Caching in HDFS: Lower Latency, Same Great Taste
In-memory Caching in HDFS: Lower Latency, Same Great TasteIn-memory Caching in HDFS: Lower Latency, Same Great Taste
In-memory Caching in HDFS: Lower Latency, Same Great Taste
 
Drupal performance
Drupal performanceDrupal performance
Drupal performance
 

More from Colin Charles

Differences between MariaDB 10.3 & MySQL 8.0
Differences between MariaDB 10.3 & MySQL 8.0Differences between MariaDB 10.3 & MySQL 8.0
Differences between MariaDB 10.3 & MySQL 8.0
Colin Charles
 
What is MariaDB Server 10.3?
What is MariaDB Server 10.3?What is MariaDB Server 10.3?
What is MariaDB Server 10.3?
Colin Charles
 
Databases in the hosted cloud
Databases in the hosted cloud Databases in the hosted cloud
Databases in the hosted cloud
Colin Charles
 
The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it! The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it!
Colin Charles
 
Databases in the Hosted Cloud
Databases in the Hosted CloudDatabases in the Hosted Cloud
Databases in the Hosted Cloud
Colin Charles
 
Best practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability TutorialBest practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability Tutorial
Colin Charles
 
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
Colin Charles
 
Capacity planning for your data stores
Capacity planning for your data storesCapacity planning for your data stores
Capacity planning for your data stores
Colin Charles
 
Forking Successfully - or is a branch better?
Forking Successfully - or is a branch better?Forking Successfully - or is a branch better?
Forking Successfully - or is a branch better?
Colin Charles
 
The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016
Colin Charles
 

More from Colin Charles (10)

Differences between MariaDB 10.3 & MySQL 8.0
Differences between MariaDB 10.3 & MySQL 8.0Differences between MariaDB 10.3 & MySQL 8.0
Differences between MariaDB 10.3 & MySQL 8.0
 
What is MariaDB Server 10.3?
What is MariaDB Server 10.3?What is MariaDB Server 10.3?
What is MariaDB Server 10.3?
 
Databases in the hosted cloud
Databases in the hosted cloud Databases in the hosted cloud
Databases in the hosted cloud
 
The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it! The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it!
 
Databases in the Hosted Cloud
Databases in the Hosted CloudDatabases in the Hosted Cloud
Databases in the Hosted Cloud
 
Best practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability TutorialBest practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability Tutorial
 
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)
 
Capacity planning for your data stores
Capacity planning for your data storesCapacity planning for your data stores
Capacity planning for your data stores
 
Forking Successfully - or is a branch better?
Forking Successfully - or is a branch better?Forking Successfully - or is a branch better?
Forking Successfully - or is a branch better?
 
The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016
 

Recently uploaded

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 

Recently uploaded (20)

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Artificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic WarfareArtificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic Warfare
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 

Tuning Linux for your database FLOSSUK 2016

  • 1. Tuning Linux for your database Colin Charles,Team MariaDB, MariaDB Corporation colin@mariadb.com / byte@bytebot.net http://bytebot.net/blog/ | @bytebot on Twitter FLOSSUK 2016, London 16 March 2016 1
  • 2. whoami • Work on MariaDB at MariaDB Corporation (SkySQL Ab) • Merged with Monty Program Ab, makers of MariaDB • Formerly MySQL AB (exit: Sun Microsystems) • Past lives include Fedora Project (FESCO), OpenOffice.org • MySQL Community Contributor of theYear Award winner 2014 2
  • 3. Agenda • Hardware or cloud • I/O • Filesystems • Memory • CPU • Network • Resources 3
  • 4. Operating System • Focus of this talk & tools: Linux • But distributions make a difference too • like RHEL6 has tuned but Ubuntu doesn’t • Distribution versions can make a difference too • RHEL5: ktune; RHEL6: tuned; RHEL7: tuned throughput-performance 4
  • 5. Servers • When you buy servers... do you think in terms of CPU, then memory, then I/O? • Database servers clearly love I/O more than memory more than CPU • queries! scans of large tables? index scans with random I/O? BI? • Durability requires each transaction commit to be flushed to disk - i.e. call fsync() 5
  • 6. I/O • What kind of storage do you use? • SAS or SATA? • Ethernet (NAS, DRBD) • SSD • Fusion IO (NVMFS) 6
  • 7. Schedulers (I/O Elevators) • cfq: default, great for slower storage (SATA) • noop: low latency storage (SSD) • deadline: multi-process apps • Most database workloads benefit from the deadline scheduler (goal is after all to minimise seeks, prioritise process I/O) 7
  • 8. Elevators continued • Boot time: elevator=deadline •echo “deadline” > /sys/class/ block/sdaN/queue/scheduler • http://dimitrik.free.fr/blog/archives/2012/01/ mysql-performance-linux-io.html • http://www.mysqlperformanceblog.com/ 2009/01/30/linux-schedulers-in-tpcc-like- benchmark/ 8
  • 9. Sample elevator results 9 1 thread tps 1 thread ioutil% 8 threads tps 8 threads ioutil% deadline* 15000 <1% 19100 <1% deadline 6850 32% 15250 45% cfq 5880 45% 1240 94%
  • 10. File systems • Choices: ext4, XFS • mount options: • ext4: rw, nosuid, noatime, data=ordered, nobarrier • xfs: nobarrier • Turn off I/O barriers • http://lwn.net/Articles/ 283161/ • Separate files (data/logs/ undo/etc.) or not? • Don’t forget flash backed write caches or battery backed write cache on RAID cards (way more fsync/s) • iostat -dmx <interval> is your friend 10
  • 11. SSD & Flash • SSD • much more IOPS than traditional disks • lower latency • make sure you have a new RAID controller • Flash • more IOPS than SSD • very low latency • InnoDB doublewrite buffer -- MariaDB 10.0, Percona Server 5.6 (Fusion IO), MySQL 5.7 11
  • 12. RAID • RAID0 - fast writes, reads, no redundancy • RAID1 - slower writes, fast reads • RAID5 - slow for random writes, fast for sequential writes, fast reads too but slow recovery • RAID10 - fast reads & writes 12
  • 13. LVM • Why? • easily expand disk • snapshot backups • Why not? • 2-3% performance penalty • Snapshot penalties 13
  • 14. Memory • The more RAM the better -> reduces I/O requirements • Use ECC RAM • NUMA - “swap insanity” • http://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the- numa-architecture/ •numactl --interleave=all mysqld & • Twitter patch/Percona Server 5.6 • numa_interleave option • innodb_buffer_pool_populate - NUMA decisions to be made when buffer cache is clean 14
  • 15. Why does MySQL need memory? • Per-session buffers • sort buffer, temp tables • Metadata/locking • index statistics, table definitions • Caching data • decrease reads: faster response time • decrease random writes: queues write in cache; perform more sequential writes 15
  • 16. Where is memory used? • filesystem cache • binary / relay logs • MyISAM data relies on filesystem cache • MySQL cache: • innodb_buffer_pool_size (pages), MyISAM’s key_buffer_size (indexes) • Per session buffers •sort_buffer_size, join_buffer_size, read_buffer_size, tmp_table_size •free -m / SHOW ENGINE INNODB STATUS 16
  • 17. Huge pages • 2M (4M even) pages vs 4K standard Linux page •vm.nr_hugepages=8192 • MySQL benefits from hugepages, MongoDB doesn’t 17
  • 18. Swappiness • Controls how aggressively the system reclaims “mapped” memory • default: 60, vm.swappiness=0 is ok (but not for newer kernels!) • decrease: aggressive reclaiming of unmapped pagecache memory • increase: aggressive swapping of mapped memory • 3.5 kernels change behaviour (and this was backported to RHEL 2.6.32-303) 18
  • 19. CPU • default is ondemand, you should turn it to performance generally •echo "performance" > /sys/devices/ system/cpu/cpu0/cpufreq/ scaling_governor • disable powersave mode • cpufrequtils is where it’s at • look at BIOS: turbo mode/C-state/OS power control 19
  • 20. Network • Tools: tc, dropwatch • Use gigabit Ethernet (more bandwidth/ throughput, less latency) • Can also trunk/bond for HA 20
  • 21. Network II • ARP filter - prevents ARP flux •echo 1 > /proc/sys/net/ipv4/ conf/all/arp_filter • MTU size at 9,000 - “jumbo frames” • sar -n DEV 21
  • 22. Network III • allow more connections to queue up: echo 4096 > / proc/sys/net/ipv4/tcp_max_syn_backlog • /etc/sysctl.conf - net.ipv4.tcp_max_syn_backlog = 4096 • increase kernel packet buffer: net.core.netdev_max_backlog = 4096 • increase socket connection wait queue: net.core.somaxconn = 4096 • Reduce TCP timeout that comes after closing socket (default 1m): net.ipv4.tcp_fin_timeout = 30 22
  • 23. Network IV • Fully synchronous replication, ala, Galera Cluster? (NDBCluster too) • Fastest as your slowest node • evs.send_window + evs.user_send_window (something large e.g. 512) 23
  • 24. Resource Limits • ulimits / /etc/security/limits.conf - they need to be high/sensible over the defaults • -f (file size): unlimited • -t (cpu time): unlimited • -v (virtmem): unlimited • -n (open files): 64000 • -m (memsize): unlimited • -u (processes/threads): 32000 24
  • 25. Tune that database • Focus on database design - find slow, crappy queries and fix them (eg. mysql: pt- query-digest) • Reduce locking or time waiting - always benchmark your application continually • Don’t forget /etc/my.cnf (and appropriate friends) 25
  • 26. KVM/Xen/Virtualization • Most of your raw database performance comes from being hardware-centric • KVM at device level you can turn off caching • Native AIO used over threaded nowadays • VMWare: http://kb.vmware.com/selfservice/ microsites/search.do? language=en_US&cmd=displayKC&externalI d=1008542 26
  • 27. Containers • Kubernetes - vitess.io rolls out w/o issue, CoreOS, Docker • Remember when benchmarking, you have to look at local vs remote benchmarking (networking can add overhead) • 1 thread (up to 25% difference), but average seems to be 15% loss for r/w workloads; remotely? 32% is possible 27
  • 28. EC2 • EC2 - instance limitations • EBS - unpredictable I/O performance, use RAID10 or RAID0 • RDS - similar performance between EBS RAID10 MySQL & RDS • choice of MySQL 5.7 (and older) or MariaDB Server 10.0 or PostgreSQL 28
  • 29. Why MariaDB/Percona Server • Threadpool • NUMA interleaving • Numerous performance fixes in XtraDB like fast InnoDB restarts, buffer pool warming, etc. • Can run (supported) MariaDB Galera Cluster/Percona XtraDB Cluster 29
  • 30. Hardware overall • Test everything • Sometimes NIC’s are a bad batch or the driver is buggy (RHEL5 + some Broadcom NICs drop packets under high load - bnx2 before 2.6.33) • Sometimes you get a bad batch of disk 30
  • 31. Hadoop/Hbase • running this in EC2 is problematic • make sure name resolution works • enable bonding • networking: hadoop uses big buffers (64MB blocks by default) • net.ipv4.tcp_rmem = 32768 436600 4194304 • net.ipv4.tcp_wmem = 32768 436600 4194304 • http://docs.hortonworks.com/HDPDocuments/HDP2/ HDP-2.1.3/bk_cluster-planning-guide/content/ch_hardware- recommendations.html 31
  • 32. Other databases • MongoDB: http://docs.mongodb.org/manual/ administration/production-notes/ • PostgreSQL: http://wiki.postgresql.org/wiki/ Tuning_Your_PostgreSQL_Server • Neo4j: http://docs.neo4j.org/chunked/stable/ configuration-linux-notes.html • Riak: http://docs.basho.com/riak/1.4.0/ cookbooks/Linux-Performance-Tuning/ 32
  • 33. SELinux, firewalls, security • The easy way is to “disable SELinux” • The reality is you should learn it • Its just like a firewall that you really should configure 33
  • 34. Tools • ps • vmstat 1 • iostat • top • free • sar (sar -n DEV 1) • gdb • tcpdump • strace • oprofile • htop • MySQL only: Percona Toolkit (pt-diskstats, pt- summary, pt-query-digest, etc.) 34
  • 35. Benchmarking • sysbench • OLTP test, use tables with 20M rows and 20M transactions, check 1-128 threads/run • LinkBench • Yahoo! Cloud Serving Benchmark • https://github.com/ brianfrankcooper/ YCSB • Google’s PerfKit Benchmarker • https://github.com/ GoogleCloudPlatfor m/ PerfKitBenchmarke r 35
  • 38. Thanks/Q&A Colin Charles, colin@mariadb.com | byte@bytebot.net http://bytebot.net/blog/ | @bytebot slides: slideshare.net/bytebot/ Download MariaDB and give it a try: http://mariadb.org/ 38